-- Logs begin at Sun 2022-07-24 19:02:18 UTC, end at Sun 2022-07-24 20:19:52 UTC. -- Jul 24 19:39:53.455468 dv-cinder-850449-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 24 19:39:53.463144 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jul 24 19:39:53 2022] *** Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: nodename: dv-cinder-850449-2-iscsi Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: machine: x86_64 Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: clock source: unix Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: pcre jit disabled Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: detected number of CPU cores: 4 Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: current working directory: / Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: detected binary path: /usr/bin/uwsgi-core Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: your processes number limit is 47680 Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: your memory page size is 4096 bytes Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: detected max file descriptor number: 2048 Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: lock engine: pthread robust mutexes Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: thunder lock: enabled Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 24 19:39:53.467714 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 24 19:39:53.511204 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: Python main interpreter initialized at 0x556e788859c0 Jul 24 19:39:53.511204 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: python threads support enabled Jul 24 19:39:53.511204 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: your server socket listen backlog is limited to 100 connections Jul 24 19:39:53.511204 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: your mercy for graceful operations on workers is 90 seconds Jul 24 19:39:53.511574 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: mapped 403077 bytes (393 KB) for 2 cores Jul 24 19:39:53.511719 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: *** Operational MODE: preforking *** Jul 24 19:39:53.511985 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: *** uWSGI is running in multiple interpreter mode *** Jul 24 19:39:53.511985 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: spawned uWSGI master process (pid: 86792) Jul 24 19:39:53.512202 dv-cinder-850449-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 24 19:39:53.516695 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: spawned uWSGI worker 1 (pid: 86806, cores: 1) Jul 24 19:39:53.516695 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: spawned uWSGI worker 2 (pid: 86807, cores: 1) Jul 24 19:39:53.516695 dv-cinder-850449-2-iscsi devstack@keystone.service[86792]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 24 19:39:59.029651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 19:39:59.029651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: from cryptography.utils import int_from_bytes Jul 24 19:39:59.030455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 19:39:59.030455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: from cryptography.utils import int_from_bytes Jul 24 19:39:59.030455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 19:39:59.030455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: from cryptography.utils import int_from_bytes Jul 24 19:39:59.030976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 19:39:59.030976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: from cryptography.utils import int_from_bytes Jul 24 19:39:59.080218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 24 19:39:59.080394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 24 19:39:59.080498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 24 19:39:59.080574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 24 19:39:59.080666 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] command line args: [] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 24 19:39:59.080732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] command line args: [] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 24 19:39:59.080952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 24 19:39:59.081049 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 24 19:39:59.081158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ================================================================================ {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 24 19:39:59.081243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ================================================================================ {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 24 19:39:59.081414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] admin_token = **** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.081490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] admin_token = **** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.081661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] config_dir = [] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.081750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] config_dir = [] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.081851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.081948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.082026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] config_source = [] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.082134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] config_source = [] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.082264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] debug = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.082358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] debug = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.082439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.082620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.082718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.082892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.082986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.083057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.083211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.083291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.083423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.083725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.083828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] list_limit = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.083828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] log_config_append = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.083943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] list_limit = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] log_config_append = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] log_dir = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] log_file = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] log_dir = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] log_file = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.084940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.085024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.085230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.085306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.085431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.085521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.085592 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.085786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.085873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.085984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.086071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.086189 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.086300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.086447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.086531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.086769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.086850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.087027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.087124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.087252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.087336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.087497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.087606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.087759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.087834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.088012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.088260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.088346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.088444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] publish_errors = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.088526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.088695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.088939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.089046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.089124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.089249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] publish_errors = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.089368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.089488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.089601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.089678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.089835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] standard_threads = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.089918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.090062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.090139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.090331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.090412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.090510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.090602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] standard_threads = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.090679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] use_journal = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.090791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.090866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] use_json = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] use_stderr = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091102 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] use_syslog = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] use_journal = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] use_json = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.091800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] use_stderr = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091895 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] use_syslog = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.091977 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.092128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 19:39:59.092203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.092386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.092469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.092630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.092733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.092890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.092965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.093141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.093228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.093384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.093460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.093625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.093701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.093878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.093966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.094117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.094193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.094373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.094459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.094593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.094679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.094838 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.094925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.095073 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.095149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.095321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.095407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.095559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.095642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.095811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.095893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.096062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.096140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.096301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.096376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.096543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.096618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.096833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.096917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.097085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.097161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.097325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.097400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.097582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.097667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.097844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.097920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.098082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.098172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.098314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.098392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.098557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.098633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.098788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.098868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.099021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.099099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.099260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.099335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.099501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.099575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.099750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.099826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.099982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.100074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.100214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.100294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.100594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.100755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.100755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.100943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.101981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.102089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.102166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.102322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.102409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.102554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.102681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.102785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.102869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.103033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.103124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.103256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.103334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.103489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.103575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.103727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.103803 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.103959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.104048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.104195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.104314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.104430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.104508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.104687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.104785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.104920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.104996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.105157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.105237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.105393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.105482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.105629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.105723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.105867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.106478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.106629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.106719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.106870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.106999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.107127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.107273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.107402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.107539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.107672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.107804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.107930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.108057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.108194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.108329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.108455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.108586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.108586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.108842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.108977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.109124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.109249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.109371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.109498 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.109623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.109754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.109955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.109955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.110192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.110318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.110448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.110573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.110706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.110843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.110968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.111102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.111232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.111357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.111492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.111627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.111757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.111883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.112003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.112128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.112253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.112386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.112513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.112667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.112791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.112918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.113052 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.113186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.113312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.113312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.113555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.113555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.113793 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.113928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.114053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.114179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.114305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.114440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.114571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.114705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.114831 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.114957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.115095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.115223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.115348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.115481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.115607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.115737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.115862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.115987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.116128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.116262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.116386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.116386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.116621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.116776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.116902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.117036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.117162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.117288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.117413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.117537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.117662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.117798 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.117934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.118061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.118186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] auth.external = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.118186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.118422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.118422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.118667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.118794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] auth.external = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.118919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.119070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.119206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] auth.password = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.119206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] auth.token = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.119445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.119589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.119722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.119847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.119978 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] auth.password = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.120103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.120236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] auth.token = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.120371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.120496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.120627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.120846 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.120974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.121104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.121236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.121369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.121506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] credential.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.122178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.122309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.122463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.122595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.122727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.122855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.122981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] credential.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.123109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.123243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.123379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.123510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.123639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.123765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.123901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.124034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.124173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.124300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.124427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.124554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.124692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.124832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.124962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.125089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.125224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.125350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.125477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.125685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.125815 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.125942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.126068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.126193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.126318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] federation.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.126451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.126578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.126713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.126839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.126966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.127091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.127226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] federation.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.127353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.127493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.127620 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.127746 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.127872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.128006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.128132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.128256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.128382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.128508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.128636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.128796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.128924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.129050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.129185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.129309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.129440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.129574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.129702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.129828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.129953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.130079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.130204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.130337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.130463 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.130588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.130588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.130816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.130893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.130973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.131975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.132945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133168 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133238 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.133953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134094 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134170 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134740 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134817 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.134935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.135038 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.135177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.135278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.135429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.135515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.135681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.135760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.135919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.135996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.136161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.136251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.136396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.136474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.136635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.136770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.136904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.136994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.137188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.137287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.137408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.137509 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.137626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.137755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.137883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.138013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.138132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.138267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.138368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.138506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.138591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.138741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.138819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.138994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.139072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.139248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.139327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.139486 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.139564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.139724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.139802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.139962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.140041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.140203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.140282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.140447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.140525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.140702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.140785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.140935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.141013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.141169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.141246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.141403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.141484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.141648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.141764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.141862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.141960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.142114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.142194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.142352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.142438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.142589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.142668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.142829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.142909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.143062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.143141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.143292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.143370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.143538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.143619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.143776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.143854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.144011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.144089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.144247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.144327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.144479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.144555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.144746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.144827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.144986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.145066 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.145225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.145303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.145464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.145542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.145696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.145781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.145939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.146017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.146172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.146249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.146398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.146473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.146630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.146709 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.146882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.146967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.147121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.147199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.147354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.147432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.147588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.147677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.147826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.147904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] resource.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.148076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.148154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.148320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.148404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.148554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.148631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.148818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.148897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.149060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] resource.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.149149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.149300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.149378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.149549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.149626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.149784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.149868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.150024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.150103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.150265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] role.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.150343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.150498 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.150584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] role.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.150728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.150805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.150970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.151047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.151211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.151296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.151448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] role.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.151527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.151697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] role.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.151776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.151933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.152019 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.152177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.152253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.152424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.152502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.152690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.153273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.153450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.153529 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.153705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.153783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.153951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.154036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.154199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.154275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.154446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.154523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.154698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.154858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.154939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.155095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.155174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.155346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.155425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.155596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.155713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.155830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.156019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.156098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.156259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.156338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.156511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.156589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.156786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.156866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.157043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.157169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.157375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.157503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.157582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.157735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.157815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.157985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.158063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.158231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.158308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.158465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.158548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.158701 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.158779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.158937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.159021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.159165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.159242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.159398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] token.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.159476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.159643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.159721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.159891 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.159970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.160127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.160204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.160361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.160438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.160598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] token.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.160696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.160862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.160942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.161103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.161181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.161339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.161418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.161585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.161662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.161822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.161900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.162058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.162138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.162296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.162373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.162538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.162615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.162786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.162864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.163026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.163111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.163263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.163341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.163496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.163575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.163744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.163822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.connection = **** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.163994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.164081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.164225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.164312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.164467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.164546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.164765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.164847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.165008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.165098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.connection = **** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.165243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.165331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.165485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.165564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.165719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.165797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.165961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.166361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.166442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.166514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.166681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.166760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.166913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.167000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.167155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.167232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.167391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.167468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.167629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.167708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.167857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.167938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.168096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.168175 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.168328 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.168409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.168559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 24 19:39:59.169159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.169383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.169621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.169864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.170101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 19:39:59.170283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86807) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 24 19:39:59.199336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.199551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.199637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.199813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.199897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.200047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.208632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.209105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.237753 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.237964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.238099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.238224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.384129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.384328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.434068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.434306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.435208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.435450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.499971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.500192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.500280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.500417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.500499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.500637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.500742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.500882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.500963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.501098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.501192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.501344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.527371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.527643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.528391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.528608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.528714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.528852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.528934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.529070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.529151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.529286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.529368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.529550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.529765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.529986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.540699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.540939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.553794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.554989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.555116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.555197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.555332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86807) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 24 19:39:59.572185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.572393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 24 19:39:59.572492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.572704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 24 19:39:59.572786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.572964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 24 19:39:59.573045 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.573237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 24 19:39:59.573316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.573497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.573575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.573753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.573834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.574010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.574088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.574268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.585664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.585942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.586247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.586501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.587107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.587393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.587666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.587954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.595280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.596722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.598604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.598881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.599165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.599562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.599851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.600102 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.600191 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.600378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.600459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.600679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.600800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.600958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.601090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.601235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.601380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.601507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.601646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.601922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.602202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.602478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.602757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.603028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.627943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.629604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.629890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.630134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.631786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.632069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.635195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.635422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.635705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.637933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.638328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.638614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.643588 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.643872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.646002 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.646293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.648477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.648802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.649083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.649362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.649640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.649921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.650192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.650460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.651084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.651373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.651661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.651948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.652229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.652508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.653390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.653683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.669955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.670230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.672926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.673210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.674890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.677856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.678547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.678843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.681621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.681923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.683878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.686989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.687534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.690668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.691327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.691629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.691932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.692217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.692502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.692806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.694516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.694824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.695123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.695424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.695706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.695994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.712219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.712592 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.712886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.713154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.716356 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.716772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.717050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.717325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.721615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.723333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.723609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.725250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.726697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.726968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.727057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.727247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.727332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.727528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.727806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.730327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.730616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.730928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.733555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.733555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.745921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.746266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.746382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.746466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.746567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.746682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.746763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.746937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.747221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.747300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.747374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.747447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.747717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.747946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.767796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.768096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.771110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.771445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.774445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.774817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.776068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.776452 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.777816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.778621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.778752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.778913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.779056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.779272 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.786610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.786984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.788296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.788851 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.791887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.792189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.792382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.792508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.792699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.792869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.793025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.793263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.802260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.802551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.802719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.802833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.803023 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.803309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.810763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.811047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.811279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86807) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 24 19:39:59.811364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.811567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.811649 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.811848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.811928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.812123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.812204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.812397 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.812479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.812682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.812776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.812958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.813038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.813247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.813326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.813518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.813598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.813786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.813867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.814061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.814141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.814324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.814406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.814592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.814870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86807) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 24 19:39:59.853482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556e788859c0 pid: 86806 (default app) Jul 24 19:39:59.853932 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556e788859c0 pid: 86807 (default app) Jul 24 19:39:59.856924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578bdc-8e21-45c5-b84e-5545b769d11d None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:39:59.857225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578bdc-8e21-45c5-b84e-5545b769d11d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:39:59.857505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578bdc-8e21-45c5-b84e-5545b769d11d None None] PATH_INFO: `/v3` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:39:59.861670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Sun Jul 24 19:39:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:08.517364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0b16e-26ba-4a17-b0af-776d0f79f209 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:08.517933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0b16e-26ba-4a17-b0af-776d0f79f209 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:08.517933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0b16e-26ba-4a17-b0af-776d0f79f209 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:08.522249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:08.534811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:08.535387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:08.535387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:08.604970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: INFO dbcounter [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] Registered counter for database keystone Jul 24 19:40:08.626233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG oslo_db.sqlalchemy.engines [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86806) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 19:40:08.632584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writer thread running {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 19:40:08.811805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG passlib.handlers.bcrypt [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86806) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 24 19:40:08.812224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG passlib.handlers.bcrypt [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86806) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 24 19:40:08.867782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:08.869432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:08.996167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 24 19:40:09.181824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0b16e-26ba-4a17-b0af-776d0f79f209 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:09.251580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: INFO dbcounter [None req-faf0b16e-26ba-4a17-b0af-776d0f79f209 None None] Registered counter for database keystone Jul 24 19:40:09.271644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG oslo_db.sqlalchemy.engines [None req-faf0b16e-26ba-4a17-b0af-776d0f79f209 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86807) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 19:40:09.281359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writer thread running {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 19:40:09.414216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0b16e-26ba-4a17-b0af-776d0f79f209 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:09.418954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4d284-13c9-4651-8d24-7f86c45f0da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:09.419629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4d284-13c9-4651-8d24-7f86c45f0da4 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:09.419915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4d284-13c9-4651-8d24-7f86c45f0da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:09.420192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4d284-13c9-4651-8d24-7f86c45f0da4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:09.429104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4d284-13c9-4651-8d24-7f86c45f0da4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:09.482149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ead4d284-13c9-4651-8d24-7f86c45f0da4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 24 19:40:09.482492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4d284-13c9-4651-8d24-7f86c45f0da4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:09.492933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Sun Jul 24 19:40:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 19:40:09.498992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:09.514337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff228206-24e3-488e-94ac-e34b44b2e142 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:09.518966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44392c9d-fdc3-419f-90fc-08daf7c771fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:09.520119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44392c9d-fdc3-419f-90fc-08daf7c771fe admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:09.520409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44392c9d-fdc3-419f-90fc-08daf7c771fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:09.520711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44392c9d-fdc3-419f-90fc-08daf7c771fe admin admin] PATH_INFO: `/v3/projects/245371efe76c48ce9eb35e790603face` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:09.524873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44392c9d-fdc3-419f-90fc-08daf7c771fe admin admin] RBAC: Authorizing `identity:get_project(project_id=245371efe76c48ce9eb35e790603face)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:09.577283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WARNING keystone.common.rbac_enforcer.enforcer [None req-44392c9d-fdc3-419f-90fc-08daf7c771fe admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 24 19:40:09.577895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44392c9d-fdc3-419f-90fc-08daf7c771fe admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:09.582320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Sun Jul 24 19:40:09 2022] GET /identity/v3/projects/245371efe76c48ce9eb35e790603face => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:11.624090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82483a-9f44-4c6e-93e3-043029b5e1c6 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:11.624622 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82483a-9f44-4c6e-93e3-043029b5e1c6 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:11.624622 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82483a-9f44-4c6e-93e3-043029b5e1c6 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:11.628765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:11.634113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bd994-79f9-4f8f-a4c9-2e07c072446e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:11.634628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bd994-79f9-4f8f-a4c9-2e07c072446e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:11.634723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bd994-79f9-4f8f-a4c9-2e07c072446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:11.653551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ec4bd994-79f9-4f8f-a4c9-2e07c072446e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:11.655099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ec4bd994-79f9-4f8f-a4c9-2e07c072446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:11.663057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 24 19:40:11.691302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82483a-9f44-4c6e-93e3-043029b5e1c6 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:11.706915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82483a-9f44-4c6e-93e3-043029b5e1c6 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:11.716253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cca104-3a0c-4caa-8db4-3d1136dff7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:11.716928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cca104-3a0c-4caa-8db4-3d1136dff7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:11.717219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cca104-3a0c-4caa-8db4-3d1136dff7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:11.717502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cca104-3a0c-4caa-8db4-3d1136dff7c5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:11.725980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cca104-3a0c-4caa-8db4-3d1136dff7c5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:11.727821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cca104-3a0c-4caa-8db4-3d1136dff7c5 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:11.738210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Sun Jul 24 19:40:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 19:40:11.743974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bd994-79f9-4f8f-a4c9-2e07c072446e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:11.753738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bd994-79f9-4f8f-a4c9-2e07c072446e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:11.758200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712d0d5-fdc2-4026-8ff7-f0952fefcaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:11.759077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712d0d5-fdc2-4026-8ff7-f0952fefcaec admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:11.759363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712d0d5-fdc2-4026-8ff7-f0952fefcaec admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:11.759642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712d0d5-fdc2-4026-8ff7-f0952fefcaec admin admin] PATH_INFO: `/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:11.761071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712d0d5-fdc2-4026-8ff7-f0952fefcaec admin admin] RBAC: Authorizing `identity:get_user(user_id=3cfc6b9179e849cdbe9d2d91ab4d0972)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:11.762900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712d0d5-fdc2-4026-8ff7-f0952fefcaec admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:11.764507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Sun Jul 24 19:40:11 2022] GET /identity/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:13.889734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824c2fa-3076-44af-b3ed-4fa573914632 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:13.890417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824c2fa-3076-44af-b3ed-4fa573914632 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:13.890417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824c2fa-3076-44af-b3ed-4fa573914632 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:13.891656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:13.894799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b2f1a-5e88-4947-831c-f5e00a915977 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:13.895252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b2f1a-5e88-4947-831c-f5e00a915977 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:13.895403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b2f1a-5e88-4947-831c-f5e00a915977 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:13.897366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:13.902686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ba69c-399b-4c8c-871a-33fb7d176e5b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:13.903553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ba69c-399b-4c8c-871a-33fb7d176e5b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:13.903840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ba69c-399b-4c8c-871a-33fb7d176e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:13.904692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fedb23-f1f1-4a7b-bed4-eae45d76f4a0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:13.905001 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fedb23-f1f1-4a7b-bed4-eae45d76f4a0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:13.905508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fedb23-f1f1-4a7b-bed4-eae45d76f4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:13.922768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b1fedb23-f1f1-4a7b-bed4-eae45d76f4a0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:13.923977 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b1fedb23-f1f1-4a7b-bed4-eae45d76f4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:13.944290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG passlib.handlers.bcrypt [None req-279ba69c-399b-4c8c-871a-33fb7d176e5b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86807) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 24 19:40:13.944741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG passlib.handlers.bcrypt [None req-279ba69c-399b-4c8c-871a-33fb7d176e5b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86807) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 24 19:40:13.990698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-279ba69c-399b-4c8c-871a-33fb7d176e5b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:13.992315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-279ba69c-399b-4c8c-871a-33fb7d176e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:14.007295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 7/12] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:14.031325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fedb23-f1f1-4a7b-bed4-eae45d76f4a0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:14.055431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:14.094823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 6/13] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:14.116565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ba69c-399b-4c8c-871a-33fb7d176e5b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:14.135942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fedb23-f1f1-4a7b-bed4-eae45d76f4a0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:14.171795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9d3d5-abab-4dd0-804a-ddf6a98354f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:14.172523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9d3d5-abab-4dd0-804a-ddf6a98354f2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:14.172849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9d3d5-abab-4dd0-804a-ddf6a98354f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:14.173132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9d3d5-abab-4dd0-804a-ddf6a98354f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:14.184897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9d3d5-abab-4dd0-804a-ddf6a98354f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-f2e9d3d5-abab-4dd0-804a-ddf6a98354f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:14.196276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:14.197491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:14.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 8/14] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:40:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:40:14.203973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9d3d5-abab-4dd0-804a-ddf6a98354f2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:14.217440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ba69c-399b-4c8c-871a-33fb7d176e5b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:14.219757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:14.256444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e0e3c-2644-4a2d-a5c5-0a594f6bb2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:14.257681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e0e3c-2644-4a2d-a5c5-0a594f6bb2e5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:14.257979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e0e3c-2644-4a2d-a5c5-0a594f6bb2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:14.258264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e0e3c-2644-4a2d-a5c5-0a594f6bb2e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:14.290402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:14.309964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9d3d5-abab-4dd0-804a-ddf6a98354f2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:14.331214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e0e3c-2644-4a2d-a5c5-0a594f6bb2e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:14.332797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e0e3c-2644-4a2d-a5c5-0a594f6bb2e5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:14.346080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7822fbb-93fc-4de5-be1d-9d8fbca33580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:14.346778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7822fbb-93fc-4de5-be1d-9d8fbca33580 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:14.347070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7822fbb-93fc-4de5-be1d-9d8fbca33580 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:14.347359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7822fbb-93fc-4de5-be1d-9d8fbca33580 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:14.348133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7822fbb-93fc-4de5-be1d-9d8fbca33580 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:14.348452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7822fbb-93fc-4de5-be1d-9d8fbca33580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:14.349907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7822fbb-93fc-4de5-be1d-9d8fbca33580 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:14.363523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 9/15] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:14.369501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7822fbb-93fc-4de5-be1d-9d8fbca33580 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:14.437590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 7/16] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 24 19:40:14.443902 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e0e3c-2644-4a2d-a5c5-0a594f6bb2e5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:14.447699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:14.475780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7822fbb-93fc-4de5-be1d-9d8fbca33580 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:14.511263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0853ac-e20e-44bc-93d7-49180426b8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:14.512142 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0853ac-e20e-44bc-93d7-49180426b8ca None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:14.512492 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0853ac-e20e-44bc-93d7-49180426b8ca None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:14.512933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0853ac-e20e-44bc-93d7-49180426b8ca None admin] PATH_INFO: `/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:14.514171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0853ac-e20e-44bc-93d7-49180426b8ca None admin] RBAC: Authorizing `identity:get_user(user_id=3cfc6b9179e849cdbe9d2d91ab4d0972)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:14.515633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0853ac-e20e-44bc-93d7-49180426b8ca None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:14.517308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 10/17] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:14 2022] GET /identity/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:14.522650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0853ac-e20e-44bc-93d7-49180426b8ca None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:14.537461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:14.545368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e0e3c-2644-4a2d-a5c5-0a594f6bb2e5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:14.580453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711f8b2-489d-4795-ae75-fcf5eadd1905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:14.581519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711f8b2-489d-4795-ae75-fcf5eadd1905 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:14.581951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711f8b2-489d-4795-ae75-fcf5eadd1905 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:14.582221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711f8b2-489d-4795-ae75-fcf5eadd1905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:14.601667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711f8b2-489d-4795-ae75-fcf5eadd1905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:14.603139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711f8b2-489d-4795-ae75-fcf5eadd1905 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:14.624445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0853ac-e20e-44bc-93d7-49180426b8ca None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a711f8b2-489d-4795-ae75-fcf5eadd1905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:40:14.624620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:14.625574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:14.626433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:40:14.626433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:40:14.626433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:40:14.626433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:14.626433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 8/18] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:40:14.631716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711f8b2-489d-4795-ae75-fcf5eadd1905 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:14.661694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e98b-2e63-4bd0-ad01-db63e1e6fc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:14.662727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e98b-2e63-4bd0-ad01-db63e1e6fc19 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:14.662853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e98b-2e63-4bd0-ad01-db63e1e6fc19 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:14.663058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e98b-2e63-4bd0-ad01-db63e1e6fc19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:14.664257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e98b-2e63-4bd0-ad01-db63e1e6fc19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:14.665745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e98b-2e63-4bd0-ad01-db63e1e6fc19 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:14.667794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 11/19] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:14.673886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e98b-2e63-4bd0-ad01-db63e1e6fc19 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:14.689489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:14.733713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711f8b2-489d-4795-ae75-fcf5eadd1905 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:14.755661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:14.767165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96260ba5-29fa-4335-8194-239449431162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:14.767838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96260ba5-29fa-4335-8194-239449431162 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:14.770240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96260ba5-29fa-4335-8194-239449431162 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:14.770240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96260ba5-29fa-4335-8194-239449431162 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:14.770240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96260ba5-29fa-4335-8194-239449431162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:14.770240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96260ba5-29fa-4335-8194-239449431162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:14.770874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96260ba5-29fa-4335-8194-239449431162 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:14.780029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e98b-2e63-4bd0-ad01-db63e1e6fc19 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:14.789566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 9/20] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:14.816413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120a0d5-6625-483f-bf59-3b00d7a0bdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:14.817130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120a0d5-6625-483f-bf59-3b00d7a0bdd0 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:14.817418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120a0d5-6625-483f-bf59-3b00d7a0bdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:14.817695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120a0d5-6625-483f-bf59-3b00d7a0bdd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:14.818551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120a0d5-6625-483f-bf59-3b00d7a0bdd0 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.domain.id=default, user.id=3cfc6b9179e849cdbe9d2d91ab4d0972) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:14.818865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120a0d5-6625-483f-bf59-3b00d7a0bdd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:14.820349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120a0d5-6625-483f-bf59-3b00d7a0bdd0 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:14.828855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 12/21] 10.222.0.17 () {56 vars in 1322 bytes} [Sun Jul 24 19:40:14 2022] GET /identity/v3/role_assignments?user.id=3cfc6b9179e849cdbe9d2d91ab4d0972&scope.domain.id=default&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:17.113734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873cb71-1fe6-4340-8914-1deb0b14d158 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.114610 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873cb71-1fe6-4340-8914-1deb0b14d158 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.115255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873cb71-1fe6-4340-8914-1deb0b14d158 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.118024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 10/22] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:17.124489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c24bd2-3c34-4015-a508-9406038702d1 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.125379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c24bd2-3c34-4015-a508-9406038702d1 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.125807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c24bd2-3c34-4015-a508-9406038702d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.150236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-51c24bd2-3c34-4015-a508-9406038702d1 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:17.151685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-51c24bd2-3c34-4015-a508-9406038702d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:17.245111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 13/23] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:17.281047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873cb71-1fe6-4340-8914-1deb0b14d158 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:17.446127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873cb71-1fe6-4340-8914-1deb0b14d158 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:17.467041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:17.494399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80410-e7db-45ec-bc1a-d6ed9e570e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:17.496824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80410-e7db-45ec-bc1a-d6ed9e570e4a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.496824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80410-e7db-45ec-bc1a-d6ed9e570e4a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.496824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80410-e7db-45ec-bc1a-d6ed9e570e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.513262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80410-e7db-45ec-bc1a-d6ed9e570e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:17.515091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80410-e7db-45ec-bc1a-d6ed9e570e4a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-ecb80410-e7db-45ec-bc1a-d6ed9e570e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:40:17.542572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:17.543506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:17.544288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:40:17.544288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:40:17.544288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:40:17.544288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:17.546286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 11/24] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:17.552410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c24bd2-3c34-4015-a508-9406038702d1 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:17.573863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195e8be-e6f0-4427-be72-c2bc0895b5eb None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.574537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195e8be-e6f0-4427-be72-c2bc0895b5eb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.575567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195e8be-e6f0-4427-be72-c2bc0895b5eb None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.577184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 12/25] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:17.582444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36b57b-d098-4570-88b3-61f41a15e1d0 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.582976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36b57b-d098-4570-88b3-61f41a15e1d0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.583476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36b57b-d098-4570-88b3-61f41a15e1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.625150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-fc36b57b-d098-4570-88b3-61f41a15e1d0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:17.626704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fc36b57b-d098-4570-88b3-61f41a15e1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:17.685497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:17.710216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c24bd2-3c34-4015-a508-9406038702d1 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:17.721478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 13/26] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:17.723327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e05fc5-662e-4313-8112-b921320a01c7 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.723779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e05fc5-662e-4313-8112-b921320a01c7 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.724147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e05fc5-662e-4313-8112-b921320a01c7 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.725933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 14/27] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:17.727647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c5461-eaeb-44dc-b5ec-1c2058241f5a None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.728179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c5461-eaeb-44dc-b5ec-1c2058241f5a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.728924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c5461-eaeb-44dc-b5ec-1c2058241f5a None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.730526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 15/28] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:17.732349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59d2ea-5af4-49d8-8e36-cd004c56b1a3 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.733165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59d2ea-5af4-49d8-8e36-cd004c56b1a3 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.733666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59d2ea-5af4-49d8-8e36-cd004c56b1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.760935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ac59d2ea-5af4-49d8-8e36-cd004c56b1a3 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:17.763101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ac59d2ea-5af4-49d8-8e36-cd004c56b1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:17.777399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104fe8f-71ef-484e-86f9-b181df890d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:17.779088 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104fe8f-71ef-484e-86f9-b181df890d27 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.779593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104fe8f-71ef-484e-86f9-b181df890d27 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.780071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104fe8f-71ef-484e-86f9-b181df890d27 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.781167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104fe8f-71ef-484e-86f9-b181df890d27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:17.781765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104fe8f-71ef-484e-86f9-b181df890d27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:17.783479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104fe8f-71ef-484e-86f9-b181df890d27 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:17.803908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 14/29] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:17.805656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0c760-c8e3-49a3-b6ee-d2224e71548d None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:17.806172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0c760-c8e3-49a3-b6ee-d2224e71548d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:17.806662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0c760-c8e3-49a3-b6ee-d2224e71548d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:17.832554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-4fe0c760-c8e3-49a3-b6ee-d2224e71548d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:17.833802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4fe0c760-c8e3-49a3-b6ee-d2224e71548d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:17.853317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 16/30] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:17.854548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59d2ea-5af4-49d8-8e36-cd004c56b1a3 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:17.896619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:17.943710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 15/31] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:17.944987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0c760-c8e3-49a3-b6ee-d2224e71548d None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:17.987232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59d2ea-5af4-49d8-8e36-cd004c56b1a3 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:18.018332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:18.026357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a778f-fdad-4d86-b5d2-4de70fb1cf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:18.027047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a778f-fdad-4d86-b5d2-4de70fb1cf6c None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:18.027340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a778f-fdad-4d86-b5d2-4de70fb1cf6c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:18.027632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a778f-fdad-4d86-b5d2-4de70fb1cf6c None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:18.028408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a778f-fdad-4d86-b5d2-4de70fb1cf6c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:18.029842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a778f-fdad-4d86-b5d2-4de70fb1cf6c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:18.057013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0c760-c8e3-49a3-b6ee-d2224e71548d None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:18.093524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee7852-9772-4911-9a18-81858510a950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:18.094203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee7852-9772-4911-9a18-81858510a950 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:18.094622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee7852-9772-4911-9a18-81858510a950 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:18.094906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee7852-9772-4911-9a18-81858510a950 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:18.095627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee7852-9772-4911-9a18-81858510a950 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:18.097140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee7852-9772-4911-9a18-81858510a950 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:18.260295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 17/32] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Jul 24 19:40:17 2022] POST /identity/v3/roles => generated 219 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:18.262157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a778f-fdad-4d86-b5d2-4de70fb1cf6c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:18.347847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 16/33] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:40:17 2022] POST /identity/v3/projects => generated 301 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:18.348916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee7852-9772-4911-9a18-81858510a950 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:18.410600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:18.427227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a778f-fdad-4d86-b5d2-4de70fb1cf6c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:18.480146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1b67b-4e0c-43e5-8d1c-c4370a4dac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:18.481705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1b67b-4e0c-43e5-8d1c-c4370a4dac6d None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:18.482138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1b67b-4e0c-43e5-8d1c-c4370a4dac6d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:18.482558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1b67b-4e0c-43e5-8d1c-c4370a4dac6d None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:18.483815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1b67b-4e0c-43e5-8d1c-c4370a4dac6d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:18.487228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1b67b-4e0c-43e5-8d1c-c4370a4dac6d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:18.562160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee7852-9772-4911-9a18-81858510a950 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:18.571046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 18/34] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Jul 24 19:40:18 2022] POST /identity/v3/roles => generated 225 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:18.601933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:18.617745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f845b1-ad19-4dee-b7e3-b24f8c6bd179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:18.618833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f845b1-ad19-4dee-b7e3-b24f8c6bd179 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:18.619302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f845b1-ad19-4dee-b7e3-b24f8c6bd179 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:18.619725 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f845b1-ad19-4dee-b7e3-b24f8c6bd179 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:18.637693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f845b1-ad19-4dee-b7e3-b24f8c6bd179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-f7f845b1-ad19-4dee-b7e3-b24f8c6bd179 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:18.646059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:18.647154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:18.648114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:18.655557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 17/35] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:40:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:18.661871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1b67b-4e0c-43e5-8d1c-c4370a4dac6d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:18.733083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa399df8-4696-4fac-85cd-578e20e6e0ef None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:18.733635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa399df8-4696-4fac-85cd-578e20e6e0ef None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:18.734058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa399df8-4696-4fac-85cd-578e20e6e0ef None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:18.736159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 18/36] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:18.742302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5170c-ed4f-4c68-bf5d-50719f2edaa9 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:18.742918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5170c-ed4f-4c68-bf5d-50719f2edaa9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:18.743417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5170c-ed4f-4c68-bf5d-50719f2edaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:18.763685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-84a5170c-ed4f-4c68-bf5d-50719f2edaa9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:18.765730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-84a5170c-ed4f-4c68-bf5d-50719f2edaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:18.811604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1b67b-4e0c-43e5-8d1c-c4370a4dac6d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:18.857306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4a3d6-8e74-4a50-bd21-49a2238de78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:18.857916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4a3d6-8e74-4a50-bd21-49a2238de78f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:18.858204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4a3d6-8e74-4a50-bd21-49a2238de78f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:18.858489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4a3d6-8e74-4a50-bd21-49a2238de78f None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:18.859254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4a3d6-8e74-4a50-bd21-49a2238de78f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:18.859568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4a3d6-8e74-4a50-bd21-49a2238de78f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:18.860990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4a3d6-8e74-4a50-bd21-49a2238de78f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:18.870458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 19/37] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:18.872470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccad6d9-32d8-49dd-b1cc-611f82cb0dd9 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:18.872997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccad6d9-32d8-49dd-b1cc-611f82cb0dd9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:18.873424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccad6d9-32d8-49dd-b1cc-611f82cb0dd9 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:18.873824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 19/38] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:18.875201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 20/39] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:18.879331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4a3d6-8e74-4a50-bd21-49a2238de78f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:18.880628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b5e20-fa44-4550-9dce-7a26dd77e0c8 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:18.880971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b5e20-fa44-4550-9dce-7a26dd77e0c8 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:18.881254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b5e20-fa44-4550-9dce-7a26dd77e0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:18.894680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:18.898517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-0f5b5e20-fa44-4550-9dce-7a26dd77e0c8 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:18.899727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0f5b5e20-fa44-4550-9dce-7a26dd77e0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:18.969354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 21/40] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:18.970552 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b5e20-fa44-4550-9dce-7a26dd77e0c8 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:18.990831 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4a3d6-8e74-4a50-bd21-49a2238de78f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:19.005101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:19.025776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4248c77-2087-483d-bf53-9027a50299ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:19.026669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4248c77-2087-483d-bf53-9027a50299ad None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:19.026959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4248c77-2087-483d-bf53-9027a50299ad None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:19.027240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4248c77-2087-483d-bf53-9027a50299ad None admin] PATH_INFO: `/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:19.029434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4248c77-2087-483d-bf53-9027a50299ad None admin] RBAC: Authorizing `identity:get_user(user_id=3cfc6b9179e849cdbe9d2d91ab4d0972)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:19.030915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4248c77-2087-483d-bf53-9027a50299ad None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:19.032567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 20/41] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:18 2022] GET /identity/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:19.033366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4248c77-2087-483d-bf53-9027a50299ad None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:19.072076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b5e20-fa44-4550-9dce-7a26dd77e0c8 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:19.114221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:19.138351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4248c77-2087-483d-bf53-9027a50299ad None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:19.176472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd606b-90b2-4419-b3e1-2e65f2f0ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:19.177302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd606b-90b2-4419-b3e1-2e65f2f0ce2f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:19.177591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd606b-90b2-4419-b3e1-2e65f2f0ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:19.177866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd606b-90b2-4419-b3e1-2e65f2f0ce2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:19.179176 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd606b-90b2-4419-b3e1-2e65f2f0ce2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:19.180632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd606b-90b2-4419-b3e1-2e65f2f0ce2f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:19.182307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 21/42] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:19.183101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd606b-90b2-4419-b3e1-2e65f2f0ce2f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:19.214883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5493d-1e3b-497b-bcee-0d8b6f5bd4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:19.216187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5493d-1e3b-497b-bcee-0d8b6f5bd4f6 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:19.216477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5493d-1e3b-497b-bcee-0d8b6f5bd4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:19.216790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5493d-1e3b-497b-bcee-0d8b6f5bd4f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:19.217549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5493d-1e3b-497b-bcee-0d8b6f5bd4f6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:19.219073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5493d-1e3b-497b-bcee-0d8b6f5bd4f6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:19.281402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd606b-90b2-4419-b3e1-2e65f2f0ce2f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:19.296389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 22/43] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Jul 24 19:40:18 2022] POST /identity/v3/roles => generated 223 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:19.297337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5493d-1e3b-497b-bcee-0d8b6f5bd4f6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:19.316894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dff2d5-a783-4df0-ac47-bc2352a85481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:19.317803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dff2d5-a783-4df0-ac47-bc2352a85481 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:19.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dff2d5-a783-4df0-ac47-bc2352a85481 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:19.318370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dff2d5-a783-4df0-ac47-bc2352a85481 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:19.319664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dff2d5-a783-4df0-ac47-bc2352a85481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:19.321230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dff2d5-a783-4df0-ac47-bc2352a85481 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:19.323082 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 22/44] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:19.329855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dff2d5-a783-4df0-ac47-bc2352a85481 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:19.348359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:19.440875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:19.447369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5493d-1e3b-497b-bcee-0d8b6f5bd4f6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:19.453754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dff2d5-a783-4df0-ac47-bc2352a85481 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:19.489246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafcb91-6ae3-4813-8b35-cb0b0772540b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:19.489441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafcb91-6ae3-4813-8b35-cb0b0772540b None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:19.490051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafcb91-6ae3-4813-8b35-cb0b0772540b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:19.490051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafcb91-6ae3-4813-8b35-cb0b0772540b None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:19.491602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafcb91-6ae3-4813-8b35-cb0b0772540b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:19.492478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafcb91-6ae3-4813-8b35-cb0b0772540b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:19.501963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c5fdf-d15b-40a8-8afb-5d581ab58683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:19.502892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c5fdf-d15b-40a8-8afb-5d581ab58683 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:19.503184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c5fdf-d15b-40a8-8afb-5d581ab58683 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:19.503473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c5fdf-d15b-40a8-8afb-5d581ab58683 None admin] PATH_INFO: `/v3/domains/default/users/3cfc6b9179e849cdbe9d2d91ab4d0972/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:19.505616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c5fdf-d15b-40a8-8afb-5d581ab58683 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3cfc6b9179e849cdbe9d2d91ab4d0972, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:19.507104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c5fdf-d15b-40a8-8afb-5d581ab58683 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:19.577802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 23/45] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:40:19 2022] POST /identity/v3/projects => generated 312 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:19.599280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 23/46] 10.222.0.17 () {58 vars in 1361 bytes} [Sun Jul 24 19:40:19 2022] PUT /identity/v3/domains/default/users/3cfc6b9179e849cdbe9d2d91ab4d0972/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 19:40:21.619232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ef703-8b2f-4156-b625-d5c600f2b2c8 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:21.619729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ef703-8b2f-4156-b625-d5c600f2b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:21.619808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ef703-8b2f-4156-b625-d5c600f2b2c8 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:21.621338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 24/47] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:21.626861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c0466-45ce-44c5-8a34-15c5a5614308 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:21.627273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c0466-45ce-44c5-8a34-15c5a5614308 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:21.627583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c0466-45ce-44c5-8a34-15c5a5614308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:21.635889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c0c71-de4b-428f-b86d-a4006c7580d6 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:21.636186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c0c71-de4b-428f-b86d-a4006c7580d6 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:21.636471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c0c71-de4b-428f-b86d-a4006c7580d6 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:21.638409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 25/48] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:21.643442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38860a2c-3b94-4eb8-832d-84d50ccfc2fa None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:21.643735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38860a2c-3b94-4eb8-832d-84d50ccfc2fa None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:21.644016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38860a2c-3b94-4eb8-832d-84d50ccfc2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:21.645143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-6f0c0466-45ce-44c5-8a34-15c5a5614308 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:21.646662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6f0c0466-45ce-44c5-8a34-15c5a5614308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:21.660086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-38860a2c-3b94-4eb8-832d-84d50ccfc2fa None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:21.661327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-38860a2c-3b94-4eb8-832d-84d50ccfc2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:21.714896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 24/49] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:21.727240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 26/50] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:21.737323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c0466-45ce-44c5-8a34-15c5a5614308 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:21.740246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:21.750735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38860a2c-3b94-4eb8-832d-84d50ccfc2fa None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:21.828680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:21.839012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c0466-45ce-44c5-8a34-15c5a5614308 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:21.849980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38860a2c-3b94-4eb8-832d-84d50ccfc2fa None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:21.873205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cb5a9-fe35-49ef-87b7-c5227cf17a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:21.874007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb5a9-fe35-49ef-87b7-c5227cf17a19 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:21.874296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb5a9-fe35-49ef-87b7-c5227cf17a19 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:21.876742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb5a9-fe35-49ef-87b7-c5227cf17a19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:21.884126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8bb51-177f-49ef-83d2-4e075b63e0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:21.885042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8bb51-177f-49ef-83d2-4e075b63e0fd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:21.885360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8bb51-177f-49ef-83d2-4e075b63e0fd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:21.885646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8bb51-177f-49ef-83d2-4e075b63e0fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:21.886920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8bb51-177f-49ef-83d2-4e075b63e0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:21.889082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8bb51-177f-49ef-83d2-4e075b63e0fd None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:21.890829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 27/51] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:21.896595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8bb51-177f-49ef-83d2-4e075b63e0fd None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:22.000744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cb5a9-fe35-49ef-87b7-c5227cf17a19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:22.007605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8bb51-177f-49ef-83d2-4e075b63e0fd None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-3c0cb5a9-fe35-49ef-87b7-c5227cf17a19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:22.009789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:22.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:22.011716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 25/52] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:40:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:22.016894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cb5a9-fe35-49ef-87b7-c5227cf17a19 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:22.045135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7350db6-dd83-4695-88ce-797521aa5727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:22.045817 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350db6-dd83-4695-88ce-797521aa5727 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:22.046099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350db6-dd83-4695-88ce-797521aa5727 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:22.046380 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350db6-dd83-4695-88ce-797521aa5727 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:22.047125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7350db6-dd83-4695-88ce-797521aa5727 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:22.048584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7350db6-dd83-4695-88ce-797521aa5727 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:22.081545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:22.117694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cb5a9-fe35-49ef-87b7-c5227cf17a19 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:22.142137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:22.148903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 28/53] 10.222.0.17 () {60 vars in 1140 bytes} [Sun Jul 24 19:40:21 2022] POST /identity/v3/projects => generated 298 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:40:22.151063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a37f78-662f-4001-9201-a155065f1715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:22.151795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a37f78-662f-4001-9201-a155065f1715 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:22.152088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a37f78-662f-4001-9201-a155065f1715 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:22.152373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a37f78-662f-4001-9201-a155065f1715 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:22.153189 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a37f78-662f-4001-9201-a155065f1715 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:22.153507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a37f78-662f-4001-9201-a155065f1715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:22.154950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a37f78-662f-4001-9201-a155065f1715 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:22.164556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 26/54] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:22.171442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7350db6-dd83-4695-88ce-797521aa5727 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:22.279542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7350db6-dd83-4695-88ce-797521aa5727 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:22.317552 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb04fe4-467c-4e04-9c26-cb1187225b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:22.318587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb04fe4-467c-4e04-9c26-cb1187225b53 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:22.318937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb04fe4-467c-4e04-9c26-cb1187225b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:22.319295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb04fe4-467c-4e04-9c26-cb1187225b53 None admin] PATH_INFO: `/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:22.320586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb04fe4-467c-4e04-9c26-cb1187225b53 None admin] RBAC: Authorizing `identity:get_user(user_id=3cfc6b9179e849cdbe9d2d91ab4d0972)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:22.322151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb04fe4-467c-4e04-9c26-cb1187225b53 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:22.324017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 29/55] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:22 2022] GET /identity/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:22.329403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a37f78-662f-4001-9201-a155065f1715 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:22.431295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a37f78-662f-4001-9201-a155065f1715 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:22.457964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:22.466981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b28f82-45a4-4a68-8888-ac6db3375627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:22.467792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b28f82-45a4-4a68-8888-ac6db3375627 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:22.470037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b28f82-45a4-4a68-8888-ac6db3375627 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:22.470037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b28f82-45a4-4a68-8888-ac6db3375627 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:22.470037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b28f82-45a4-4a68-8888-ac6db3375627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:22.471042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b28f82-45a4-4a68-8888-ac6db3375627 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:22.472899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 27/56] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:22.478452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb04fe4-467c-4e04-9c26-cb1187225b53 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:22.545508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:22.579226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb04fe4-467c-4e04-9c26-cb1187225b53 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:22.614892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f8e35-6a70-4f73-9eb0-78aec02d0012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:22.615564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f8e35-6a70-4f73-9eb0-78aec02d0012 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:22.615853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f8e35-6a70-4f73-9eb0-78aec02d0012 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:22.616134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f8e35-6a70-4f73-9eb0-78aec02d0012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:22.617010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f8e35-6a70-4f73-9eb0-78aec02d0012 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.domain.id=default, user.id=3cfc6b9179e849cdbe9d2d91ab4d0972) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:22.617329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f8e35-6a70-4f73-9eb0-78aec02d0012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:22.618831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f8e35-6a70-4f73-9eb0-78aec02d0012 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:22.625509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 30/57] 10.222.0.17 () {56 vars in 1322 bytes} [Sun Jul 24 19:40:22 2022] GET /identity/v3/role_assignments?user.id=3cfc6b9179e849cdbe9d2d91ab4d0972&scope.domain.id=default&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:24.201390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80596a44-5885-4890-ac8e-b59a4f6a3883 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:24.201901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80596a44-5885-4890-ac8e-b59a4f6a3883 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:24.201985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80596a44-5885-4890-ac8e-b59a4f6a3883 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:24.203322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 28/58] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:24.208567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313126f-4fec-4c89-86f1-39d1e086ae4f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:24.208952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313126f-4fec-4c89-86f1-39d1e086ae4f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:24.209172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313126f-4fec-4c89-86f1-39d1e086ae4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:24.225470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b313126f-4fec-4c89-86f1-39d1e086ae4f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:24.226669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b313126f-4fec-4c89-86f1-39d1e086ae4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:24.291369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 31/59] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:24.316143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80596a44-5885-4890-ac8e-b59a4f6a3883 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:24.415449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80596a44-5885-4890-ac8e-b59a4f6a3883 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:24.449269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79d2ca-f2dd-4c70-a8e1-cbee31c6b0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:24.450594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79d2ca-f2dd-4c70-a8e1-cbee31c6b0e0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:24.450886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79d2ca-f2dd-4c70-a8e1-cbee31c6b0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:24.451168 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79d2ca-f2dd-4c70-a8e1-cbee31c6b0e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:24.452402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79d2ca-f2dd-4c70-a8e1-cbee31c6b0e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:24.453854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79d2ca-f2dd-4c70-a8e1-cbee31c6b0e0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:24.455548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 29/60] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:24.461014 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313126f-4fec-4c89-86f1-39d1e086ae4f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:24.519007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:24.559497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313126f-4fec-4c89-86f1-39d1e086ae4f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:24.593963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89ed2d-be0b-4e90-b882-cb01f7c80d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:24.594674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89ed2d-be0b-4e90-b882-cb01f7c80d96 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:24.594961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89ed2d-be0b-4e90-b882-cb01f7c80d96 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:24.595237 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89ed2d-be0b-4e90-b882-cb01f7c80d96 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:24.595952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89ed2d-be0b-4e90-b882-cb01f7c80d96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:24.597422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89ed2d-be0b-4e90-b882-cb01f7c80d96 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:24.672176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 32/61] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:40:24 2022] POST /identity/v3/users => generated 275 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:28.025084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e2535-cc5e-469a-8ee3-88fe3e411501 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.025634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e2535-cc5e-469a-8ee3-88fe3e411501 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.025634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e2535-cc5e-469a-8ee3-88fe3e411501 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.027248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 30/62] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:28.033355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d012d-9c96-4080-8ba9-30a422b7d963 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.033755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d012d-9c96-4080-8ba9-30a422b7d963 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.033938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d012d-9c96-4080-8ba9-30a422b7d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.062806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-bd5d012d-9c96-4080-8ba9-30a422b7d963 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:28.064360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-bd5d012d-9c96-4080-8ba9-30a422b7d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:28.112007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26f14a-a069-4768-862d-ebe323aadca0 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.112635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26f14a-a069-4768-862d-ebe323aadca0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.113134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26f14a-a069-4768-862d-ebe323aadca0 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.114729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 31/63] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:28.120216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635dd9b-7795-49bc-a303-31056a0d6ab9 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.120825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635dd9b-7795-49bc-a303-31056a0d6ab9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.121344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635dd9b-7795-49bc-a303-31056a0d6ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.139346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-7635dd9b-7795-49bc-a303-31056a0d6ab9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:28.140968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7635dd9b-7795-49bc-a303-31056a0d6ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:28.194614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 33/64] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:28.218843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049faeb2-5f63-47e7-9419-b30605812d65 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.219454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049faeb2-5f63-47e7-9419-b30605812d65 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.219935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049faeb2-5f63-47e7-9419-b30605812d65 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.221692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 34/65] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:28.229423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719f26-192b-422b-a24c-286419d9bccd None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.229423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719f26-192b-422b-a24c-286419d9bccd None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.229423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719f26-192b-422b-a24c-286419d9bccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.229713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:28.240726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 32/66] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:28.241851 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635dd9b-7795-49bc-a303-31056a0d6ab9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:28.250114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6e719f26-192b-422b-a24c-286419d9bccd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:28.251050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6e719f26-192b-422b-a24c-286419d9bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:28.300744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:28.328695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 35/67] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:28.328695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e719f26-192b-422b-a24c-286419d9bccd None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:28.352180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635dd9b-7795-49bc-a303-31056a0d6ab9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:28.389493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750482c-8c9b-404d-ac59-6f83d518d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:28.390304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750482c-8c9b-404d-ac59-6f83d518d513 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.390601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750482c-8c9b-404d-ac59-6f83d518d513 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.390882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750482c-8c9b-404d-ac59-6f83d518d513 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.403245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750482c-8c9b-404d-ac59-6f83d518d513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a750482c-8c9b-404d-ac59-6f83d518d513 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:28.411091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:28.411960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:28.412729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 33/68] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:28.413593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d9cb0-613b-435f-b4ee-b9252b77e6ef None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.413868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d9cb0-613b-435f-b4ee-b9252b77e6ef None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.414149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d9cb0-613b-435f-b4ee-b9252b77e6ef None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.415547 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 34/69] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:28.416105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d9cb0-613b-435f-b4ee-b9252b77e6ef None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:28.452117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e719f26-192b-422b-a24c-286419d9bccd None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:28.473048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:28.485325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011f90a-11c3-43b3-ac9f-4dcb233d1212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:28.486127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011f90a-11c3-43b3-ac9f-4dcb233d1212 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.486415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011f90a-11c3-43b3-ac9f-4dcb233d1212 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.486696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011f90a-11c3-43b3-ac9f-4dcb233d1212 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.497264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011f90a-11c3-43b3-ac9f-4dcb233d1212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-d011f90a-11c3-43b3-ac9f-4dcb233d1212 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:28.504912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:28.505738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:28.506560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 36/70] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:40:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:28.507340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc4b04-01ca-4fbb-ab3e-cb3d67f538c4 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.507614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc4b04-01ca-4fbb-ab3e-cb3d67f538c4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.507859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc4b04-01ca-4fbb-ab3e-cb3d67f538c4 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.509352 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 37/71] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:28.509846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc4b04-01ca-4fbb-ab3e-cb3d67f538c4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:28.526115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d9cb0-613b-435f-b4ee-b9252b77e6ef None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:28.545480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:28.561128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b11385-e10f-46af-9f4d-d70fd335470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:28.561867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b11385-e10f-46af-9f4d-d70fd335470d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.562151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b11385-e10f-46af-9f4d-d70fd335470d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.562425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b11385-e10f-46af-9f4d-d70fd335470d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.572987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b11385-e10f-46af-9f4d-d70fd335470d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-f3b11385-e10f-46af-9f4d-d70fd335470d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:28.581845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:28.582757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:28.583495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 35/72] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:40:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:28.584482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09f8cb-02a9-42ac-8ff4-157ae5b68ee4 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.584580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09f8cb-02a9-42ac-8ff4-157ae5b68ee4 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.584860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09f8cb-02a9-42ac-8ff4-157ae5b68ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.601733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-7f09f8cb-02a9-42ac-8ff4-157ae5b68ee4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:28.602933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7f09f8cb-02a9-42ac-8ff4-157ae5b68ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:28.612417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc4b04-01ca-4fbb-ab3e-cb3d67f538c4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:28.646327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c0b0e-ed91-43e4-9e1a-bccd216b51e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:28.646995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c0b0e-ed91-43e4-9e1a-bccd216b51e8 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.647285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c0b0e-ed91-43e4-9e1a-bccd216b51e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.647562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c0b0e-ed91-43e4-9e1a-bccd216b51e8 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.648312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c0b0e-ed91-43e4-9e1a-bccd216b51e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:28.648622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c0b0e-ed91-43e4-9e1a-bccd216b51e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:28.650033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c0b0e-ed91-43e4-9e1a-bccd216b51e8 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:28.658857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 38/73] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:28.659734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c0b0e-ed91-43e4-9e1a-bccd216b51e8 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:28.673463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 36/74] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:28.674946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc6215-a486-45bb-a973-27ae6c2d37db None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.675442 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc6215-a486-45bb-a973-27ae6c2d37db None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.675883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc6215-a486-45bb-a973-27ae6c2d37db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.696003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d0cc6215-a486-45bb-a973-27ae6c2d37db None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:28.697236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d0cc6215-a486-45bb-a973-27ae6c2d37db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:28.733146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:28.758781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:28.763018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c0b0e-ed91-43e4-9e1a-bccd216b51e8 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:28.764889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 37/75] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:28.778230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc6215-a486-45bb-a973-27ae6c2d37db None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:28.798338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d3ed0-a7ce-4bc4-a936-4d0ca9b816ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:28.799017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d3ed0-a7ce-4bc4-a936-4d0ca9b816ef None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.799304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d3ed0-a7ce-4bc4-a936-4d0ca9b816ef None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.799582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d3ed0-a7ce-4bc4-a936-4d0ca9b816ef None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.800334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d3ed0-a7ce-4bc4-a936-4d0ca9b816ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:28.800667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d3ed0-a7ce-4bc4-a936-4d0ca9b816ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:28.802036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d3ed0-a7ce-4bc4-a936-4d0ca9b816ef None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:28.811004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 39/76] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:28.811845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d3ed0-a7ce-4bc4-a936-4d0ca9b816ef None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:28.876389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc6215-a486-45bb-a973-27ae6c2d37db None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:28.911326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3891af-1703-4eba-841c-7c106ac81ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:28.912008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3891af-1703-4eba-841c-7c106ac81ccb None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.912296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3891af-1703-4eba-841c-7c106ac81ccb None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.912486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d3ed0-a7ce-4bc4-a936-4d0ca9b816ef None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:28.912585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3891af-1703-4eba-841c-7c106ac81ccb None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.913367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3891af-1703-4eba-841c-7c106ac81ccb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:28.913678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3891af-1703-4eba-841c-7c106ac81ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:28.915100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3891af-1703-4eba-841c-7c106ac81ccb None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:28.924393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 38/77] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:40:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:28.925265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3891af-1703-4eba-841c-7c106ac81ccb None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:28.948659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d70dcf-e810-407a-b585-8a3ba2aa859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:28.949594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d70dcf-e810-407a-b585-8a3ba2aa859c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:28.949881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d70dcf-e810-407a-b585-8a3ba2aa859c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:28.950158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d70dcf-e810-407a-b585-8a3ba2aa859c None admin] PATH_INFO: `/v3/users/fcdb4126379942f5816db31b2de8f861` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:28.959124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:28.963751 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d70dcf-e810-407a-b585-8a3ba2aa859c None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb4126379942f5816db31b2de8f861)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:28.965411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d70dcf-e810-407a-b585-8a3ba2aa859c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:28.970191 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 40/78] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:28 2022] GET /identity/v3/users/fcdb4126379942f5816db31b2de8f861 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:28.970191 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d70dcf-e810-407a-b585-8a3ba2aa859c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:28.987504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:29.028636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3891af-1703-4eba-841c-7c106ac81ccb None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:29.064592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a609b-c0cb-4e5c-b56c-e0abc7ec9a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:29.065341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a609b-c0cb-4e5c-b56c-e0abc7ec9a85 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:29.065621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a609b-c0cb-4e5c-b56c-e0abc7ec9a85 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:29.065907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a609b-c0cb-4e5c-b56c-e0abc7ec9a85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:29.068668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d70dcf-e810-407a-b585-8a3ba2aa859c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:29.075914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a609b-c0cb-4e5c-b56c-e0abc7ec9a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-472a609b-c0cb-4e5c-b56c-e0abc7ec9a85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:29.083620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:29.084550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:29.085344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 39/79] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:29.085960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a609b-c0cb-4e5c-b56c-e0abc7ec9a85 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:29.108179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4a299-4194-4ef7-90c8-136f77c51366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:29.109173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4a299-4194-4ef7-90c8-136f77c51366 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:29.109466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4a299-4194-4ef7-90c8-136f77c51366 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:29.109755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4a299-4194-4ef7-90c8-136f77c51366 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:29.110930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4a299-4194-4ef7-90c8-136f77c51366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:29.112391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4a299-4194-4ef7-90c8-136f77c51366 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:29.114053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 41/80] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:29.114833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4a299-4194-4ef7-90c8-136f77c51366 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:29.178490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:29.185118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a609b-c0cb-4e5c-b56c-e0abc7ec9a85 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:29.202019 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:29.217080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4a299-4194-4ef7-90c8-136f77c51366 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:29.218604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fb655-5d36-4e55-91f8-858093d24a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:29.219496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fb655-5d36-4e55-91f8-858093d24a1c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:29.219786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fb655-5d36-4e55-91f8-858093d24a1c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:29.220067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fb655-5d36-4e55-91f8-858093d24a1c None admin] PATH_INFO: `/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:29.221343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fb655-5d36-4e55-91f8-858093d24a1c None admin] RBAC: Authorizing `identity:get_user(user_id=3cfc6b9179e849cdbe9d2d91ab4d0972)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:29.222824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fb655-5d36-4e55-91f8-858093d24a1c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:29.224515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 40/81] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:29 2022] GET /identity/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:29.225379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fb655-5d36-4e55-91f8-858093d24a1c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:29.257977 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3382d55-ff65-484e-bd27-9bf2d2c2d4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:29.258890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3382d55-ff65-484e-bd27-9bf2d2c2d4d0 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:29.259182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3382d55-ff65-484e-bd27-9bf2d2c2d4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:29.259464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3382d55-ff65-484e-bd27-9bf2d2c2d4d0 None admin] PATH_INFO: `/v3/users/fcdb4126379942f5816db31b2de8f861` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:29.260803 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3382d55-ff65-484e-bd27-9bf2d2c2d4d0 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb4126379942f5816db31b2de8f861)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:29.262344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3382d55-ff65-484e-bd27-9bf2d2c2d4d0 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:29.264124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 42/82] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:29 2022] GET /identity/v3/users/fcdb4126379942f5816db31b2de8f861 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:29.264967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3382d55-ff65-484e-bd27-9bf2d2c2d4d0 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:29.330879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fb655-5d36-4e55-91f8-858093d24a1c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:29.371255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5cdf4-d88a-4a7f-9361-c17f6b25ea2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:29.371910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5cdf4-d88a-4a7f-9361-c17f6b25ea2a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:29.372194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5cdf4-d88a-4a7f-9361-c17f6b25ea2a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:29.372470 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5cdf4-d88a-4a7f-9361-c17f6b25ea2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:29.372973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3382d55-ff65-484e-bd27-9bf2d2c2d4d0 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:29.404286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:29.409506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d43b31-f649-4fe1-a73c-a8f3a0a004ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:29.410195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d43b31-f649-4fe1-a73c-a8f3a0a004ce None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:29.410487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d43b31-f649-4fe1-a73c-a8f3a0a004ce None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:29.410789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d43b31-f649-4fe1-a73c-a8f3a0a004ce None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:29.411557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d43b31-f649-4fe1-a73c-a8f3a0a004ce None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:29.416181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d43b31-f649-4fe1-a73c-a8f3a0a004ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:29.416181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d43b31-f649-4fe1-a73c-a8f3a0a004ce None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:29.422856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 43/83] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:29.423708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d43b31-f649-4fe1-a73c-a8f3a0a004ce None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:29.445750 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:29.456448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5cdf4-d88a-4a7f-9361-c17f6b25ea2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:29.457995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5cdf4-d88a-4a7f-9361-c17f6b25ea2a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:29.532412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d43b31-f649-4fe1-a73c-a8f3a0a004ce None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:29.562800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 41/84] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:29.563692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5cdf4-d88a-4a7f-9361-c17f6b25ea2a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:29.567498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e7f4e-01fc-4c59-a909-463472706b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:29.568104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e7f4e-01fc-4c59-a909-463472706b62 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:29.568403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e7f4e-01fc-4c59-a909-463472706b62 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:29.568713 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e7f4e-01fc-4c59-a909-463472706b62 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:29.569452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e7f4e-01fc-4c59-a909-463472706b62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:29.570912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e7f4e-01fc-4c59-a909-463472706b62 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:29.628837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:29.644660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 44/85] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:40:29 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:29.645548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e7f4e-01fc-4c59-a909-463472706b62 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:29.680198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5cdf4-d88a-4a7f-9361-c17f6b25ea2a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:29.724318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1f408-676e-45cb-9dd4-fbe67001d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:29.725033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1f408-676e-45cb-9dd4-fbe67001d18f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:29.725325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1f408-676e-45cb-9dd4-fbe67001d18f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:29.725648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1f408-676e-45cb-9dd4-fbe67001d18f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:29.732877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:29.766884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e7f4e-01fc-4c59-a909-463472706b62 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:29.807528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3622f1-c752-416b-a31a-98a890f7c910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:29.808198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3622f1-c752-416b-a31a-98a890f7c910 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:29.808491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3622f1-c752-416b-a31a-98a890f7c910 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:29.808886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3622f1-c752-416b-a31a-98a890f7c910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:29.811815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1f408-676e-45cb-9dd4-fbe67001d18f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:29.813359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1f408-676e-45cb-9dd4-fbe67001d18f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:29.887811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:29.892567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3622f1-c752-416b-a31a-98a890f7c910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:29.894983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3622f1-c752-416b-a31a-98a890f7c910 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:29.939335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 42/86] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:29.940579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1f408-676e-45cb-9dd4-fbe67001d18f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:29.982432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:30.021184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 45/87] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:30.022606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3622f1-c752-416b-a31a-98a890f7c910 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:30.057022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1f408-676e-45cb-9dd4-fbe67001d18f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:30.098687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff82cdf-b3b4-4bd7-9e69-dedda5979d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:30.099585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff82cdf-b3b4-4bd7-9e69-dedda5979d87 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:30.099924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff82cdf-b3b4-4bd7-9e69-dedda5979d87 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:30.100209 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff82cdf-b3b4-4bd7-9e69-dedda5979d87 None admin] PATH_INFO: `/v3/users/fcdb4126379942f5816db31b2de8f861` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:30.102177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff82cdf-b3b4-4bd7-9e69-dedda5979d87 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb4126379942f5816db31b2de8f861)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:30.108726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff82cdf-b3b4-4bd7-9e69-dedda5979d87 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:30.108726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 43/88] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:29 2022] GET /identity/v3/users/fcdb4126379942f5816db31b2de8f861 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:30.108726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff82cdf-b3b4-4bd7-9e69-dedda5979d87 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:30.174589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:30.180842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3622f1-c752-416b-a31a-98a890f7c910 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:30.229041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198f5c3-163e-474a-b257-7c15241f5671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:30.230454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198f5c3-163e-474a-b257-7c15241f5671 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:30.232002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198f5c3-163e-474a-b257-7c15241f5671 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:30.232483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198f5c3-163e-474a-b257-7c15241f5671 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:30.234447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198f5c3-163e-474a-b257-7c15241f5671 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:30.236873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198f5c3-163e-474a-b257-7c15241f5671 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:30.240068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 46/89] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:30 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:30.241165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198f5c3-163e-474a-b257-7c15241f5671 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:30.274371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff82cdf-b3b4-4bd7-9e69-dedda5979d87 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:30.295707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:30.319581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443a201-f094-4a27-82e9-34093b13802c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:30.320458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443a201-f094-4a27-82e9-34093b13802c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:30.320778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443a201-f094-4a27-82e9-34093b13802c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:30.321059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443a201-f094-4a27-82e9-34093b13802c None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:30.322268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443a201-f094-4a27-82e9-34093b13802c None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:30.323718 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443a201-f094-4a27-82e9-34093b13802c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:30.326862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 44/90] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:30 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:30.331657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443a201-f094-4a27-82e9-34093b13802c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:30.382516 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198f5c3-163e-474a-b257-7c15241f5671 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:30.421223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce29ed7-fc38-4b1e-99ce-b5adbea576ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:30.422121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce29ed7-fc38-4b1e-99ce-b5adbea576ac None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:30.422412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce29ed7-fc38-4b1e-99ce-b5adbea576ac None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:30.422695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce29ed7-fc38-4b1e-99ce-b5adbea576ac None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:30.423970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce29ed7-fc38-4b1e-99ce-b5adbea576ac None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:30.425544 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce29ed7-fc38-4b1e-99ce-b5adbea576ac None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:30.427248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 47/91] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:30 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:30.433066 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce29ed7-fc38-4b1e-99ce-b5adbea576ac None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:30.463445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:30.469144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443a201-f094-4a27-82e9-34093b13802c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:30.508605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d2a46-be35-4d55-b0f6-abf2e61bebe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:30.509646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d2a46-be35-4d55-b0f6-abf2e61bebe0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:30.510117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d2a46-be35-4d55-b0f6-abf2e61bebe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:30.510577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d2a46-be35-4d55-b0f6-abf2e61bebe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:30.539171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce29ed7-fc38-4b1e-99ce-b5adbea576ac None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:30.550237 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:30.576785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588bc3e-117b-4836-9dce-8ad1437c9051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:30.577436 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588bc3e-117b-4836-9dce-8ad1437c9051 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:30.577731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588bc3e-117b-4836-9dce-8ad1437c9051 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:30.578015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588bc3e-117b-4836-9dce-8ad1437c9051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:30.578873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588bc3e-117b-4836-9dce-8ad1437c9051 None admin] RBAC: Adding query filter params (role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be, user.id=fcdb4126379942f5816db31b2de8f861) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:30.579194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588bc3e-117b-4836-9dce-8ad1437c9051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:30.580667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588bc3e-117b-4836-9dce-8ad1437c9051 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:30.587374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 48/92] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:30 2022] GET /identity/v3/role_assignments?user.id=fcdb4126379942f5816db31b2de8f861&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:30.588962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588bc3e-117b-4836-9dce-8ad1437c9051 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:30.592630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d2a46-be35-4d55-b0f6-abf2e61bebe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:30.594616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d2a46-be35-4d55-b0f6-abf2e61bebe0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:30.701819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588bc3e-117b-4836-9dce-8ad1437c9051 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:30.710562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:30.716995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 45/93] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:30.718370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d2a46-be35-4d55-b0f6-abf2e61bebe0 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:30.740715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d3d31-e043-4d53-8011-3d6170f1db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:30.741412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d3d31-e043-4d53-8011-3d6170f1db6e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:30.741702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d3d31-e043-4d53-8011-3d6170f1db6e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:30.742196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d3d31-e043-4d53-8011-3d6170f1db6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:30.743079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d3d31-e043-4d53-8011-3d6170f1db6e None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be, user.id=3cfc6b9179e849cdbe9d2d91ab4d0972) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:30.743402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d3d31-e043-4d53-8011-3d6170f1db6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:30.745260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d3d31-e043-4d53-8011-3d6170f1db6e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:30.752328 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 49/94] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:30 2022] GET /identity/v3/role_assignments?user.id=3cfc6b9179e849cdbe9d2d91ab4d0972&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:30.753390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d3d31-e043-4d53-8011-3d6170f1db6e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:30.796535 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:30.843900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d2a46-be35-4d55-b0f6-abf2e61bebe0 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:30.876333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d3d31-e043-4d53-8011-3d6170f1db6e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:30.885300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a28d3-9eae-4af5-a78a-e69a442ab19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:30.886391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a28d3-9eae-4af5-a78a-e69a442ab19f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:30.886904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a28d3-9eae-4af5-a78a-e69a442ab19f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:30.887370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a28d3-9eae-4af5-a78a-e69a442ab19f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:30.888593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a28d3-9eae-4af5-a78a-e69a442ab19f None admin] RBAC: Adding query filter params (role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be, user.id=fcdb4126379942f5816db31b2de8f861) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:30.889162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a28d3-9eae-4af5-a78a-e69a442ab19f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:30.891057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a28d3-9eae-4af5-a78a-e69a442ab19f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:30.901239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 46/95] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:30 2022] GET /identity/v3/role_assignments?user.id=fcdb4126379942f5816db31b2de8f861&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:30.924955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca62aad-98be-4bc1-95e5-50adc9e34772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:30.926274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca62aad-98be-4bc1-95e5-50adc9e34772 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:30.926783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca62aad-98be-4bc1-95e5-50adc9e34772 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:30.927254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca62aad-98be-4bc1-95e5-50adc9e34772 None admin] PATH_INFO: `/v3/projects/c5fbbd3f0070448e8fedfb89f9ba0e83` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:30.929005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca62aad-98be-4bc1-95e5-50adc9e34772 None admin] RBAC: Authorizing `identity:get_project(project_id=c5fbbd3f0070448e8fedfb89f9ba0e83)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:30.930840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca62aad-98be-4bc1-95e5-50adc9e34772 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:30.934105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 50/96] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:30 2022] GET /identity/v3/projects/c5fbbd3f0070448e8fedfb89f9ba0e83 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:30.941968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a28d3-9eae-4af5-a78a-e69a442ab19f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:31.033034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:31.079933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a28d3-9eae-4af5-a78a-e69a442ab19f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:31.123117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749297-f354-40cc-9591-c823df2522a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:31.123812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749297-f354-40cc-9591-c823df2522a9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:31.124106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749297-f354-40cc-9591-c823df2522a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:31.124394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749297-f354-40cc-9591-c823df2522a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:31.125299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749297-f354-40cc-9591-c823df2522a9 None admin] RBAC: Adding query filter params (role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=c5fbbd3f0070448e8fedfb89f9ba0e83, user.id=fcdb4126379942f5816db31b2de8f861) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:31.125623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749297-f354-40cc-9591-c823df2522a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:31.127147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749297-f354-40cc-9591-c823df2522a9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:31.136784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 47/97] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:30 2022] GET /identity/v3/role_assignments?user.id=fcdb4126379942f5816db31b2de8f861&scope.project.id=c5fbbd3f0070448e8fedfb89f9ba0e83&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:32.286609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555f8cc-adbc-406a-b074-353617b8908f None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:32.287115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555f8cc-adbc-406a-b074-353617b8908f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:32.287223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555f8cc-adbc-406a-b074-353617b8908f None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:32.288862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 51/98] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:32.294828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcf40f-edf9-417d-b09c-acd5be30c491 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:32.295231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcf40f-edf9-417d-b09c-acd5be30c491 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:32.295467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcf40f-edf9-417d-b09c-acd5be30c491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:32.317708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-07bcf40f-edf9-417d-b09c-acd5be30c491 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:32.318950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-07bcf40f-edf9-417d-b09c-acd5be30c491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:32.401468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 48/99] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:32.434298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555f8cc-adbc-406a-b074-353617b8908f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:32.601006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555f8cc-adbc-406a-b074-353617b8908f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:32.603566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:32.645104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17ba0d-76d1-4f73-ac37-d5156e0b0bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:32.645983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17ba0d-76d1-4f73-ac37-d5156e0b0bc7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:32.646279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17ba0d-76d1-4f73-ac37-d5156e0b0bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:32.646561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17ba0d-76d1-4f73-ac37-d5156e0b0bc7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:32.665596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17ba0d-76d1-4f73-ac37-d5156e0b0bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-5c17ba0d-76d1-4f73-ac37-d5156e0b0bc7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:32.674564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:32.675401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:32.676127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 52/100] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:32.682596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcf40f-edf9-417d-b09c-acd5be30c491 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:32.796676 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:32.866279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcf40f-edf9-417d-b09c-acd5be30c491 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:32.917473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3f77f-3d38-4088-b623-e67cc601f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:32.918725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3f77f-3d38-4088-b623-e67cc601f4ae None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:32.919017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3f77f-3d38-4088-b623-e67cc601f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:32.919297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3f77f-3d38-4088-b623-e67cc601f4ae None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:32.920062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3f77f-3d38-4088-b623-e67cc601f4ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:32.920370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3f77f-3d38-4088-b623-e67cc601f4ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:32.921780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3f77f-3d38-4088-b623-e67cc601f4ae None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:32.930657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 49/101] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:32.936294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17ba0d-76d1-4f73-ac37-d5156e0b0bc7 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:32.938415 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e807eb-5c11-4dca-af68-a035173bbdbd None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:32.938714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e807eb-5c11-4dca-af68-a035173bbdbd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:32.938997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e807eb-5c11-4dca-af68-a035173bbdbd None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:32.940361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 50/102] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:32.958409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b7418-5885-40a7-b7b9-648bcf981ae9 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:32.958927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b7418-5885-40a7-b7b9-648bcf981ae9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:32.959348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b7418-5885-40a7-b7b9-648bcf981ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:32.989674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-015b7418-5885-40a7-b7b9-648bcf981ae9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:32.991118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-015b7418-5885-40a7-b7b9-648bcf981ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:33.151046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 51/103] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:33.162813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17ba0d-76d1-4f73-ac37-d5156e0b0bc7 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:33.187533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b7418-5885-40a7-b7b9-648bcf981ae9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:33.217369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011066-1357-4a82-b032-a5bd09c86fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:33.218270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011066-1357-4a82-b032-a5bd09c86fda None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:33.218564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011066-1357-4a82-b032-a5bd09c86fda None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:33.218849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011066-1357-4a82-b032-a5bd09c86fda None admin] PATH_INFO: `/v3/users/922a719c2f784a3a8905fd5ed41c6304` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:33.239902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42011066-1357-4a82-b032-a5bd09c86fda None admin] RBAC: Authorizing `identity:get_user(user_id=922a719c2f784a3a8905fd5ed41c6304)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:33.241700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42011066-1357-4a82-b032-a5bd09c86fda None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:33.243465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 53/104] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:32 2022] GET /identity/v3/users/922a719c2f784a3a8905fd5ed41c6304 => generated 289 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:33.249488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011066-1357-4a82-b032-a5bd09c86fda None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:33.289007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:33.338390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:33.410458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011066-1357-4a82-b032-a5bd09c86fda None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:33.468443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5169ef6-15a1-4c62-a822-3a1d5fc9a826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:33.469993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169ef6-15a1-4c62-a822-3a1d5fc9a826 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:33.470493 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169ef6-15a1-4c62-a822-3a1d5fc9a826 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:33.470976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169ef6-15a1-4c62-a822-3a1d5fc9a826 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:33.513909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b7418-5885-40a7-b7b9-648bcf981ae9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:33.594575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5169ef6-15a1-4c62-a822-3a1d5fc9a826 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:33.596397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5169ef6-15a1-4c62-a822-3a1d5fc9a826 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:33.608165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbbf7f-f90b-4dca-a11c-8cad34676d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:33.609071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbbf7f-f90b-4dca-a11c-8cad34676d79 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:33.609368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbbf7f-f90b-4dca-a11c-8cad34676d79 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:33.609652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbbf7f-f90b-4dca-a11c-8cad34676d79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:33.610907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbbf7f-f90b-4dca-a11c-8cad34676d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:33.612387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbbf7f-f90b-4dca-a11c-8cad34676d79 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:33.616670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 52/105] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:33.619507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbbf7f-f90b-4dca-a11c-8cad34676d79 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:33.646344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:33.765453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbbf7f-f90b-4dca-a11c-8cad34676d79 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:33.785498 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:33.804398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c88af3-57f0-4797-a354-2f659033398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:33.805296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c88af3-57f0-4797-a354-2f659033398a None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:33.805687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c88af3-57f0-4797-a354-2f659033398a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:33.806072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c88af3-57f0-4797-a354-2f659033398a None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:33.806930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c88af3-57f0-4797-a354-2f659033398a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:33.808569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c88af3-57f0-4797-a354-2f659033398a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:33.820334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 54/106] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 24 19:40:33.821957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc1e02-ae89-4d17-a9e5-94c56ff384d3 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:33.822412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc1e02-ae89-4d17-a9e5-94c56ff384d3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:33.822833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc1e02-ae89-4d17-a9e5-94c56ff384d3 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:33.824317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 55/107] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:33.831629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc1e02-ae89-4d17-a9e5-94c56ff384d3 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:33.911624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 53/108] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:40:33 2022] POST /identity/v3/projects => generated 302 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:40:33.915222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85c57f-1c7e-4c4a-8dc7-2f1637903c40 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:33.916952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85c57f-1c7e-4c4a-8dc7-2f1637903c40 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:33.917754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85c57f-1c7e-4c4a-8dc7-2f1637903c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:33.947122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-6f85c57f-1c7e-4c4a-8dc7-2f1637903c40 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:33.948899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6f85c57f-1c7e-4c4a-8dc7-2f1637903c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:34.006385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc1e02-ae89-4d17-a9e5-94c56ff384d3 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:34.059057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:34.079002 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 54/109] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:34.081214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeed8f9-f025-457f-bf83-e3a9953982f0 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.082214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeed8f9-f025-457f-bf83-e3a9953982f0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.083162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeed8f9-f025-457f-bf83-e3a9953982f0 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.085240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 55/110] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:34.087083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178acb-a31d-4e72-91b1-8cd2c5668db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:34.088260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178acb-a31d-4e72-91b1-8cd2c5668db5 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.088773 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178acb-a31d-4e72-91b1-8cd2c5668db5 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.089557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178acb-a31d-4e72-91b1-8cd2c5668db5 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.092398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb3f4-3bdf-4b08-adf6-d98cc9df401b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.092921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb3f4-3bdf-4b08-adf6-d98cc9df401b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.093380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb3f4-3bdf-4b08-adf6-d98cc9df401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.093921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9178acb-a31d-4e72-91b1-8cd2c5668db5 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:34.095485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9178acb-a31d-4e72-91b1-8cd2c5668db5 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:34.097701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 56/111] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:33 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:34.104443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178acb-a31d-4e72-91b1-8cd2c5668db5 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:34.113477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ebcfb3f4-3bdf-4b08-adf6-d98cc9df401b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:34.114959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ebcfb3f4-3bdf-4b08-adf6-d98cc9df401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:34.201522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 56/112] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:34.202418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfb3f4-3bdf-4b08-adf6-d98cc9df401b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:34.228445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178acb-a31d-4e72-91b1-8cd2c5668db5 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:34.237041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:34.266816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1b765-0c06-46a4-80ea-b57831f9b339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:34.267797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1b765-0c06-46a4-80ea-b57831f9b339 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.268089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1b765-0c06-46a4-80ea-b57831f9b339 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.268374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1b765-0c06-46a4-80ea-b57831f9b339 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.269261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1b765-0c06-46a4-80ea-b57831f9b339 None admin] RBAC: Adding query filter params (role.id=86259b0903754e10b0da0ff74bbfa7f8, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be, user.id=922a719c2f784a3a8905fd5ed41c6304) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:34.269588 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1b765-0c06-46a4-80ea-b57831f9b339 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:34.271054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1b765-0c06-46a4-80ea-b57831f9b339 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:34.277223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 57/113] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:34 2022] GET /identity/v3/role_assignments?user.id=922a719c2f784a3a8905fd5ed41c6304&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=86259b0903754e10b0da0ff74bbfa7f8 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:34.287105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4ed54-503f-4158-9fee-ea740280ca3e None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.287105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4ed54-503f-4158-9fee-ea740280ca3e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.287105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4ed54-503f-4158-9fee-ea740280ca3e None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.287105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 58/114] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:34.287105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4ed54-503f-4158-9fee-ea740280ca3e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:34.332227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfb3f4-3bdf-4b08-adf6-d98cc9df401b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:34.363716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:34.381950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4233b8-9da8-46bd-bde1-d0c1a2ef781b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:34.383497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4233b8-9da8-46bd-bde1-d0c1a2ef781b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.384135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4233b8-9da8-46bd-bde1-d0c1a2ef781b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.384758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4233b8-9da8-46bd-bde1-d0c1a2ef781b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.398960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4233b8-9da8-46bd-bde1-d0c1a2ef781b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-9b4233b8-9da8-46bd-bde1-d0c1a2ef781b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:34.407618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:34.408556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:34.409377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:34.411920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 57/115] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:34.413580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be152f-76fd-4020-bfca-db35adf8513e None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.414138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be152f-76fd-4020-bfca-db35adf8513e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.414609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be152f-76fd-4020-bfca-db35adf8513e None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.416238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 58/116] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:34.418135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d189a-0cd1-428c-8f25-d487223ff3da None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.418713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d189a-0cd1-428c-8f25-d487223ff3da None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.419193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d189a-0cd1-428c-8f25-d487223ff3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.435568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4ed54-503f-4158-9fee-ea740280ca3e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:34.442761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-168d189a-0cd1-428c-8f25-d487223ff3da None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:34.444442 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-168d189a-0cd1-428c-8f25-d487223ff3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:34.472833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4d1f2-81c8-4c97-ba42-1f71a7106f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:34.473680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4d1f2-81c8-4c97-ba42-1f71a7106f45 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.473966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4d1f2-81c8-4c97-ba42-1f71a7106f45 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.474238 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4d1f2-81c8-4c97-ba42-1f71a7106f45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.485570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4d1f2-81c8-4c97-ba42-1f71a7106f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-ccd4d1f2-81c8-4c97-ba42-1f71a7106f45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:34.493793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:34.494655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:34.495521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 59/117] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:40:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:34.496702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4d1f2-81c8-4c97-ba42-1f71a7106f45 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:34.527738 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 59/118] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:34.529609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0147faa-5603-48fb-a3f9-aa228396efc9 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.530102 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0147faa-5603-48fb-a3f9-aa228396efc9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.530620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0147faa-5603-48fb-a3f9-aa228396efc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.558455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c0147faa-5603-48fb-a3f9-aa228396efc9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:34.561123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c0147faa-5603-48fb-a3f9-aa228396efc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:34.580406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:34.622371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4d1f2-81c8-4c97-ba42-1f71a7106f45 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:34.638990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:34.651976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 60/119] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:34.653104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0147faa-5603-48fb-a3f9-aa228396efc9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:34.662517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37776543-8fe6-4387-8e27-856796e15507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:34.663537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37776543-8fe6-4387-8e27-856796e15507 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.664061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37776543-8fe6-4387-8e27-856796e15507 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.664563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37776543-8fe6-4387-8e27-856796e15507 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.665703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37776543-8fe6-4387-8e27-856796e15507 None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:34.666303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37776543-8fe6-4387-8e27-856796e15507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:34.668410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37776543-8fe6-4387-8e27-856796e15507 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:34.679052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 60/120] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:34.680131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37776543-8fe6-4387-8e27-856796e15507 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:34.777940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0147faa-5603-48fb-a3f9-aa228396efc9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:34.815062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37776543-8fe6-4387-8e27-856796e15507 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:34.817027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec210b5f-da44-4437-8f1f-789c98e3e32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:34.817986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec210b5f-da44-4437-8f1f-789c98e3e32c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.818516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec210b5f-da44-4437-8f1f-789c98e3e32c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.819037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec210b5f-da44-4437-8f1f-789c98e3e32c None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.820200 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec210b5f-da44-4437-8f1f-789c98e3e32c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:34.820787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec210b5f-da44-4437-8f1f-789c98e3e32c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:34.822388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec210b5f-da44-4437-8f1f-789c98e3e32c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:34.831801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 61/121] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:40:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:34.832998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec210b5f-da44-4437-8f1f-789c98e3e32c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:34.846077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:34.856291 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa80b0-8749-4a21-9aa3-d1861e28a77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:34.857135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa80b0-8749-4a21-9aa3-d1861e28a77a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:34.857424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa80b0-8749-4a21-9aa3-d1861e28a77a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:34.857702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa80b0-8749-4a21-9aa3-d1861e28a77a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:34.872855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa80b0-8749-4a21-9aa3-d1861e28a77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-27aa80b0-8749-4a21-9aa3-d1861e28a77a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:34.882640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:34.883465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:34.884213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:34.888861 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 61/122] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:34.891678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa80b0-8749-4a21-9aa3-d1861e28a77a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:34.938512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:34.975226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec210b5f-da44-4437-8f1f-789c98e3e32c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.020846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa80b0-8749-4a21-9aa3-d1861e28a77a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.021501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11065251-0c61-4ab3-95f4-a4fe61bfb652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.022732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11065251-0c61-4ab3-95f4-a4fe61bfb652 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.023349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11065251-0c61-4ab3-95f4-a4fe61bfb652 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.023847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11065251-0c61-4ab3-95f4-a4fe61bfb652 None admin] PATH_INFO: `/v3/users/fcdb4126379942f5816db31b2de8f861` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.025304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11065251-0c61-4ab3-95f4-a4fe61bfb652 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb4126379942f5816db31b2de8f861)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:35.027071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11065251-0c61-4ab3-95f4-a4fe61bfb652 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:35.030108 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 62/123] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:34 2022] GET /identity/v3/users/fcdb4126379942f5816db31b2de8f861 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:35.031351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11065251-0c61-4ab3-95f4-a4fe61bfb652 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:35.062514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5286be4-69ef-4b39-8a73-a3917b6e7ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.063386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5286be4-69ef-4b39-8a73-a3917b6e7ba9 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.063676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5286be4-69ef-4b39-8a73-a3917b6e7ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.064008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5286be4-69ef-4b39-8a73-a3917b6e7ba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.080084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5286be4-69ef-4b39-8a73-a3917b6e7ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-e5286be4-69ef-4b39-8a73-a3917b6e7ba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:35.090985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:35.091866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:35.092627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 62/124] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:40:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:35.093289 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5286be4-69ef-4b39-8a73-a3917b6e7ba9 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:35.162611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:35.188384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11065251-0c61-4ab3-95f4-a4fe61bfb652 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.264437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:35.272155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf2032-cdde-4cc8-b00f-c80828109c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.273674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf2032-cdde-4cc8-b00f-c80828109c9a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.274262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf2032-cdde-4cc8-b00f-c80828109c9a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.274700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf2032-cdde-4cc8-b00f-c80828109c9a None admin] PATH_INFO: `/v3/users/fcdb4126379942f5816db31b2de8f861` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.276354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf2032-cdde-4cc8-b00f-c80828109c9a None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb4126379942f5816db31b2de8f861)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:35.277576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5286be4-69ef-4b39-8a73-a3917b6e7ba9 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.280578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf2032-cdde-4cc8-b00f-c80828109c9a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:35.284057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 63/125] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:35 2022] GET /identity/v3/users/fcdb4126379942f5816db31b2de8f861 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:35.285147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf2032-cdde-4cc8-b00f-c80828109c9a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:35.343275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc714867-540c-4d51-b240-937f96677079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.344036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc714867-540c-4d51-b240-937f96677079 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.344477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc714867-540c-4d51-b240-937f96677079 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.344987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc714867-540c-4d51-b240-937f96677079 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.347649 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc714867-540c-4d51-b240-937f96677079 None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:35.348139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc714867-540c-4d51-b240-937f96677079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:35.350361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc714867-540c-4d51-b240-937f96677079 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:35.370029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 63/126] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:35.372234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc714867-540c-4d51-b240-937f96677079 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:35.469795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf2032-cdde-4cc8-b00f-c80828109c9a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.501270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc714867-540c-4d51-b240-937f96677079 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.532075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9de845-32c5-4b23-9407-a1788b023f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.533075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9de845-32c5-4b23-9407-a1788b023f4f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.533373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9de845-32c5-4b23-9407-a1788b023f4f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.533657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9de845-32c5-4b23-9407-a1788b023f4f None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.534948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9de845-32c5-4b23-9407-a1788b023f4f None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:35.536429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9de845-32c5-4b23-9407-a1788b023f4f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:35.538664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 64/127] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:35 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:35.539535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9de845-32c5-4b23-9407-a1788b023f4f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:35.568048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5ba6f-2218-4653-a8c5-a56d7aa28e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.569183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5ba6f-2218-4653-a8c5-a56d7aa28e51 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.570653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5ba6f-2218-4653-a8c5-a56d7aa28e51 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.571121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5ba6f-2218-4653-a8c5-a56d7aa28e51 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.571789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:35.574788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5ba6f-2218-4653-a8c5-a56d7aa28e51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:35.575521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5ba6f-2218-4653-a8c5-a56d7aa28e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:35.581191 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5ba6f-2218-4653-a8c5-a56d7aa28e51 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:35.597661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 64/128] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:35.599000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5ba6f-2218-4653-a8c5-a56d7aa28e51 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:35.649922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:35.691231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9de845-32c5-4b23-9407-a1788b023f4f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.728155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0336b-1d83-42af-a990-e5b777b00d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.729994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0336b-1d83-42af-a990-e5b777b00d97 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.730509 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0336b-1d83-42af-a990-e5b777b00d97 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.730984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0336b-1d83-42af-a990-e5b777b00d97 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.732736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0336b-1d83-42af-a990-e5b777b00d97 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:35.734409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0336b-1d83-42af-a990-e5b777b00d97 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:35.737392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 65/129] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:35 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:35.738660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0336b-1d83-42af-a990-e5b777b00d97 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:35.785275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5ba6f-2218-4653-a8c5-a56d7aa28e51 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.822595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f05f61-d9cd-48fc-9304-2ed4f5f61692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.823498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f05f61-d9cd-48fc-9304-2ed4f5f61692 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.823783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f05f61-d9cd-48fc-9304-2ed4f5f61692 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.824060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f05f61-d9cd-48fc-9304-2ed4f5f61692 None admin] PATH_INFO: `/v3/users/fcdb4126379942f5816db31b2de8f861` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.825388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f05f61-d9cd-48fc-9304-2ed4f5f61692 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb4126379942f5816db31b2de8f861)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:35.826882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f05f61-d9cd-48fc-9304-2ed4f5f61692 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:35.828566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 65/130] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:35 2022] GET /identity/v3/users/fcdb4126379942f5816db31b2de8f861 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:35.829408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f05f61-d9cd-48fc-9304-2ed4f5f61692 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:35.834897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:35.856420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0336b-1d83-42af-a990-e5b777b00d97 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.896138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3314c0f-aea1-48c1-b6fa-31df96f1ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.897318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3314c0f-aea1-48c1-b6fa-31df96f1ddb3 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.897779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3314c0f-aea1-48c1-b6fa-31df96f1ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.898197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3314c0f-aea1-48c1-b6fa-31df96f1ddb3 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/users/fcdb4126379942f5816db31b2de8f861/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.900547 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3314c0f-aea1-48c1-b6fa-31df96f1ddb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=04a4d4b6451449af851d7f25e1f9c6be, user_id=fcdb4126379942f5816db31b2de8f861, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:35.902184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3314c0f-aea1-48c1-b6fa-31df96f1ddb3 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:35.923761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:35.935922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f05f61-d9cd-48fc-9304-2ed4f5f61692 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:35.970324 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae06414-3a02-4dad-a9b4-0c4f1b64eaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:35.971140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae06414-3a02-4dad-a9b4-0c4f1b64eaf4 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:35.971428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae06414-3a02-4dad-a9b4-0c4f1b64eaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:35.971718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae06414-3a02-4dad-a9b4-0c4f1b64eaf4 None admin] PATH_INFO: `/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:35.972926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae06414-3a02-4dad-a9b4-0c4f1b64eaf4 None admin] RBAC: Authorizing `identity:get_user(user_id=3cfc6b9179e849cdbe9d2d91ab4d0972)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:35.974388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae06414-3a02-4dad-a9b4-0c4f1b64eaf4 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:35.976022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 66/131] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:35 2022] GET /identity/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:35.977337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae06414-3a02-4dad-a9b4-0c4f1b64eaf4 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:36.009706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 66/132] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:40:35 2022] PUT /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/users/fcdb4126379942f5816db31b2de8f861/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:36.010556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3314c0f-aea1-48c1-b6fa-31df96f1ddb3 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:36.164800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3314c0f-aea1-48c1-b6fa-31df96f1ddb3 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:36.217608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae06414-3a02-4dad-a9b4-0c4f1b64eaf4 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:36.220203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4988fb-de69-43bd-b736-4ba623f2bd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:36.221426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4988fb-de69-43bd-b736-4ba623f2bd48 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.221667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4988fb-de69-43bd-b736-4ba623f2bd48 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.221934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4988fb-de69-43bd-b736-4ba623f2bd48 None admin] PATH_INFO: `/v3/projects/c5fbbd3f0070448e8fedfb89f9ba0e83` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.223648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4988fb-de69-43bd-b736-4ba623f2bd48 None admin] RBAC: Authorizing `identity:get_project(project_id=c5fbbd3f0070448e8fedfb89f9ba0e83)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:36.225197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4988fb-de69-43bd-b736-4ba623f2bd48 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:36.226994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 67/133] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:36 2022] GET /identity/v3/projects/c5fbbd3f0070448e8fedfb89f9ba0e83 => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:36.227789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4988fb-de69-43bd-b736-4ba623f2bd48 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:36.241924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:36.268277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da05303-8ea1-482e-ba3d-a3fcfddb201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:36.270500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da05303-8ea1-482e-ba3d-a3fcfddb201a None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.271061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da05303-8ea1-482e-ba3d-a3fcfddb201a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.271546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da05303-8ea1-482e-ba3d-a3fcfddb201a None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/users/fcdb4126379942f5816db31b2de8f861/roles/31a26f4cc3674ca9bdbfc0be860a4318` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.274167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da05303-8ea1-482e-ba3d-a3fcfddb201a None admin] RBAC: Authorizing `identity:create_grant(project_id=04a4d4b6451449af851d7f25e1f9c6be, user_id=fcdb4126379942f5816db31b2de8f861, role_id=31a26f4cc3674ca9bdbfc0be860a4318)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:36.276345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da05303-8ea1-482e-ba3d-a3fcfddb201a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:36.347561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4988fb-de69-43bd-b736-4ba623f2bd48 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:36.383243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 67/134] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:40:35 2022] PUT /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/users/fcdb4126379942f5816db31b2de8f861/roles/31a26f4cc3674ca9bdbfc0be860a4318 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:36.384782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da05303-8ea1-482e-ba3d-a3fcfddb201a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:36.387297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a163495-5010-4efc-a57f-02d86c4d32c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:36.388256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a163495-5010-4efc-a57f-02d86c4d32c3 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.388729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a163495-5010-4efc-a57f-02d86c4d32c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.389429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a163495-5010-4efc-a57f-02d86c4d32c3 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.392103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a163495-5010-4efc-a57f-02d86c4d32c3 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:36.393766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a163495-5010-4efc-a57f-02d86c4d32c3 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:36.396075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 68/135] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:36 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:36.399301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849f995-1482-4f18-a0de-498dbd485cfe None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.399602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849f995-1482-4f18-a0de-498dbd485cfe None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.407482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849f995-1482-4f18-a0de-498dbd485cfe None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.409118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 69/136] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:36 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:36.417771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1902afb-e18b-41c1-80f4-ad82dbd930ee None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.418105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1902afb-e18b-41c1-80f4-ad82dbd930ee None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.418398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1902afb-e18b-41c1-80f4-ad82dbd930ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.419727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:36.445691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d1902afb-e18b-41c1-80f4-ad82dbd930ee None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:36.446977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d1902afb-e18b-41c1-80f4-ad82dbd930ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:36.544661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da05303-8ea1-482e-ba3d-a3fcfddb201a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:36.551623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 70/137] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:36.553178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1902afb-e18b-41c1-80f4-ad82dbd930ee None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:36.565305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:36.616536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71043262-000b-4a0a-9dbf-05bfc23a2634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:36.617845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71043262-000b-4a0a-9dbf-05bfc23a2634 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.618308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71043262-000b-4a0a-9dbf-05bfc23a2634 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.618706 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71043262-000b-4a0a-9dbf-05bfc23a2634 None admin] PATH_INFO: `/v3/projects/c5fbbd3f0070448e8fedfb89f9ba0e83/users/fcdb4126379942f5816db31b2de8f861/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.620977 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71043262-000b-4a0a-9dbf-05bfc23a2634 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5fbbd3f0070448e8fedfb89f9ba0e83, user_id=fcdb4126379942f5816db31b2de8f861, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:36.623534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71043262-000b-4a0a-9dbf-05bfc23a2634 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:36.671483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1902afb-e18b-41c1-80f4-ad82dbd930ee None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:36.685272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 68/138] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:40:36 2022] PUT /identity/v3/projects/c5fbbd3f0070448e8fedfb89f9ba0e83/users/fcdb4126379942f5816db31b2de8f861/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:36.686611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90b5ac-6a9a-46cb-a083-cb5e9541eefa None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.686970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90b5ac-6a9a-46cb-a083-cb5e9541eefa None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.687298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90b5ac-6a9a-46cb-a083-cb5e9541eefa None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.689110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 69/139] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:36.689908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90b5ac-6a9a-46cb-a083-cb5e9541eefa None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:36.737696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e16b83-33b9-4462-9315-5e40112b1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:36.738708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e16b83-33b9-4462-9315-5e40112b1c49 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.738993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e16b83-33b9-4462-9315-5e40112b1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.739269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e16b83-33b9-4462-9315-5e40112b1c49 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/users/3cfc6b9179e849cdbe9d2d91ab4d0972/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.743291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e16b83-33b9-4462-9315-5e40112b1c49 None admin] RBAC: Authorizing `identity:create_grant(project_id=04a4d4b6451449af851d7f25e1f9c6be, user_id=3cfc6b9179e849cdbe9d2d91ab4d0972, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:36.745240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e16b83-33b9-4462-9315-5e40112b1c49 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:36.815306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:36.827458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 71/140] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:40:36 2022] PUT /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/users/3cfc6b9179e849cdbe9d2d91ab4d0972/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:36.828875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d5549-6313-4ebb-8efb-5ef837f32930 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.829218 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d5549-6313-4ebb-8efb-5ef837f32930 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.829503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d5549-6313-4ebb-8efb-5ef837f32930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.842825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90b5ac-6a9a-46cb-a083-cb5e9541eefa None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:36.865914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-523d5549-6313-4ebb-8efb-5ef837f32930 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:36.867206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-523d5549-6313-4ebb-8efb-5ef837f32930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:36.883799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe333-0fcf-4b92-a52e-155d6a8d2a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:36.884670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe333-0fcf-4b92-a52e-155d6a8d2a3a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:36.885037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe333-0fcf-4b92-a52e-155d6a8d2a3a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:36.885318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe333-0fcf-4b92-a52e-155d6a8d2a3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:36.887294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbe333-0fcf-4b92-a52e-155d6a8d2a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:36.889131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbe333-0fcf-4b92-a52e-155d6a8d2a3a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:36.890997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 70/141] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:36.897084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe333-0fcf-4b92-a52e-155d6a8d2a3a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:37.016698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe333-0fcf-4b92-a52e-155d6a8d2a3a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:37.018811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:37.035545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 72/142] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:37.055791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71a014-9b4d-433a-bfd8-614e17264cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:37.056499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71a014-9b4d-433a-bfd8-614e17264cac None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:37.056821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71a014-9b4d-433a-bfd8-614e17264cac None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:37.057102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71a014-9b4d-433a-bfd8-614e17264cac None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:37.057903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71a014-9b4d-433a-bfd8-614e17264cac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:37.060009 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71a014-9b4d-433a-bfd8-614e17264cac None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:37.081118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d5549-6313-4ebb-8efb-5ef837f32930 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:37.196228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d5549-6313-4ebb-8efb-5ef837f32930 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:37.232592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451ded5-74b4-4ee3-b007-4908503ab615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:37.233430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451ded5-74b4-4ee3-b007-4908503ab615 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:37.233771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451ded5-74b4-4ee3-b007-4908503ab615 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:37.234071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451ded5-74b4-4ee3-b007-4908503ab615 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:37.247247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451ded5-74b4-4ee3-b007-4908503ab615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-9451ded5-74b4-4ee3-b007-4908503ab615 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:37.255409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:37.256325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:37.257122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 73/143] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:40:37.262267 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:37.269455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 71/144] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:40:36 2022] POST /identity/v3/users => generated 283 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:40:37.274301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451ded5-74b4-4ee3-b007-4908503ab615 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:37.352829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:37.404553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451ded5-74b4-4ee3-b007-4908503ab615 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:37.456981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576a79-f72a-451f-a7d4-48123bf41a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:37.457730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576a79-f72a-451f-a7d4-48123bf41a14 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:37.458026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576a79-f72a-451f-a7d4-48123bf41a14 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:37.458311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576a79-f72a-451f-a7d4-48123bf41a14 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:37.459081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54576a79-f72a-451f-a7d4-48123bf41a14 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:37.459406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54576a79-f72a-451f-a7d4-48123bf41a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:37.460937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54576a79-f72a-451f-a7d4-48123bf41a14 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:37.475274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 74/145] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:37.481431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71a014-9b4d-433a-bfd8-614e17264cac None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:37.593101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71a014-9b4d-433a-bfd8-614e17264cac None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:37.629359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695357c-0f70-4870-a99a-1182ce16d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:37.630183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695357c-0f70-4870-a99a-1182ce16d23b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:37.630472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695357c-0f70-4870-a99a-1182ce16d23b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:37.630753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695357c-0f70-4870-a99a-1182ce16d23b None admin] PATH_INFO: `/v3/users/922a719c2f784a3a8905fd5ed41c6304` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:37.631995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695357c-0f70-4870-a99a-1182ce16d23b None admin] RBAC: Authorizing `identity:get_user(user_id=922a719c2f784a3a8905fd5ed41c6304)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:37.633563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695357c-0f70-4870-a99a-1182ce16d23b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:37.635600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 72/146] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:37 2022] GET /identity/v3/users/922a719c2f784a3a8905fd5ed41c6304 => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:37.641671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576a79-f72a-451f-a7d4-48123bf41a14 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:37.754295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576a79-f72a-451f-a7d4-48123bf41a14 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:37.783420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:37.801855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ffb8b-cc34-4bf7-ab88-3b6d32d37f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:37.803655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffb8b-cc34-4bf7-ab88-3b6d32d37f15 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:37.803655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffb8b-cc34-4bf7-ab88-3b6d32d37f15 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:37.803655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffb8b-cc34-4bf7-ab88-3b6d32d37f15 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:37.804733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ffb8b-cc34-4bf7-ab88-3b6d32d37f15 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:37.806250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ffb8b-cc34-4bf7-ab88-3b6d32d37f15 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:37.808022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 75/147] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:37 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:37.814613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695357c-0f70-4870-a99a-1182ce16d23b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:37.893589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:37.941028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695357c-0f70-4870-a99a-1182ce16d23b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:37.987205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de66b9-eddd-4886-9e2c-5dcfc73b5def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:37.988455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de66b9-eddd-4886-9e2c-5dcfc73b5def None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:37.988988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de66b9-eddd-4886-9e2c-5dcfc73b5def None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:37.989472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de66b9-eddd-4886-9e2c-5dcfc73b5def None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/users/922a719c2f784a3a8905fd5ed41c6304/roles/86259b0903754e10b0da0ff74bbfa7f8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:37.991679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de66b9-eddd-4886-9e2c-5dcfc73b5def None admin] RBAC: Authorizing `identity:create_grant(project_id=04a4d4b6451449af851d7f25e1f9c6be, user_id=922a719c2f784a3a8905fd5ed41c6304, role_id=86259b0903754e10b0da0ff74bbfa7f8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:37.993371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de66b9-eddd-4886-9e2c-5dcfc73b5def None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:38.095090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 73/148] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:40:37 2022] PUT /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/users/922a719c2f784a3a8905fd5ed41c6304/roles/86259b0903754e10b0da0ff74bbfa7f8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:38.536609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb65f7d-579d-4038-a338-299645a20812 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:38.537001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb65f7d-579d-4038-a338-299645a20812 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:38.537269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb65f7d-579d-4038-a338-299645a20812 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:38.538820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 76/149] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:38.544765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2872e-892d-418f-be0c-ec7af7992a86 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:38.545237 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2872e-892d-418f-be0c-ec7af7992a86 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:38.545317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2872e-892d-418f-be0c-ec7af7992a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:38.567917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3ac2872e-892d-418f-be0c-ec7af7992a86 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:38.569155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3ac2872e-892d-418f-be0c-ec7af7992a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:38.675460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 74/150] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:38.722238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb65f7d-579d-4038-a338-299645a20812 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:38.852726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb65f7d-579d-4038-a338-299645a20812 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:38.896225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771cdb0-23e3-4753-a4fd-044835303a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:38.897077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771cdb0-23e3-4753-a4fd-044835303a21 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:38.897368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771cdb0-23e3-4753-a4fd-044835303a21 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:38.897648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771cdb0-23e3-4753-a4fd-044835303a21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:38.910219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771cdb0-23e3-4753-a4fd-044835303a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-1771cdb0-23e3-4753-a4fd-044835303a21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:38.918458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:38.919287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:38.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 77/151] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:38.928363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2872e-892d-418f-be0c-ec7af7992a86 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:39.017303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:39.072034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2872e-892d-418f-be0c-ec7af7992a86 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:39.116740 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd442dc3-bb1e-4c4c-86f2-5643bb9d8bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:39.117425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd442dc3-bb1e-4c4c-86f2-5643bb9d8bda None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:39.117712 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd442dc3-bb1e-4c4c-86f2-5643bb9d8bda None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:39.117986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd442dc3-bb1e-4c4c-86f2-5643bb9d8bda None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:39.118739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd442dc3-bb1e-4c4c-86f2-5643bb9d8bda None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:39.119046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd442dc3-bb1e-4c4c-86f2-5643bb9d8bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:39.120420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd442dc3-bb1e-4c4c-86f2-5643bb9d8bda None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:39.131768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 75/152] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:39.137955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771cdb0-23e3-4753-a4fd-044835303a21 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:39.205194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:39.261773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771cdb0-23e3-4753-a4fd-044835303a21 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:39.302905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffa712-72ff-4e51-a054-9947d229329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:39.303856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffa712-72ff-4e51-a054-9947d229329f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:39.304146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffa712-72ff-4e51-a054-9947d229329f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:39.304421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffa712-72ff-4e51-a054-9947d229329f None admin] PATH_INFO: `/v3/users/fcdb4126379942f5816db31b2de8f861` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:39.306165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffa712-72ff-4e51-a054-9947d229329f None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb4126379942f5816db31b2de8f861)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:39.307626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffa712-72ff-4e51-a054-9947d229329f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:39.310128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 78/153] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:39 2022] GET /identity/v3/users/fcdb4126379942f5816db31b2de8f861 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:39.315940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd442dc3-bb1e-4c4c-86f2-5643bb9d8bda None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:39.483283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd442dc3-bb1e-4c4c-86f2-5643bb9d8bda None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:39.512160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:39.529638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3edf3d-7002-4adc-943c-3a5424906314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:39.530310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3edf3d-7002-4adc-943c-3a5424906314 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:39.530582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3edf3d-7002-4adc-943c-3a5424906314 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:39.530867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3edf3d-7002-4adc-943c-3a5424906314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:39.574715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d7498-a044-455f-964b-544daebfa098 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:39.575061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d7498-a044-455f-964b-544daebfa098 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:39.575288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d7498-a044-455f-964b-544daebfa098 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:39.576731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 79/154] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:39.582343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f97e5-833b-4d5a-bfa4-5e8bf86cfe18 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:39.583768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f97e5-833b-4d5a-bfa4-5e8bf86cfe18 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:39.584264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f97e5-833b-4d5a-bfa4-5e8bf86cfe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:39.617940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-0e4f97e5-833b-4d5a-bfa4-5e8bf86cfe18 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:39.620029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0e4f97e5-833b-4d5a-bfa4-5e8bf86cfe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:39.633269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3edf3d-7002-4adc-943c-3a5424906314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:39.634860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3edf3d-7002-4adc-943c-3a5424906314 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:39.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 80/155] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:39.786596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f97e5-833b-4d5a-bfa4-5e8bf86cfe18 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:39.803087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 76/156] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:39.809719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3edf3d-7002-4adc-943c-3a5424906314 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:39.852535 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:39.880582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:39.987578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f97e5-833b-4d5a-bfa4-5e8bf86cfe18 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:40.024541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3edf3d-7002-4adc-943c-3a5424906314 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:40.051231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fbb78-5834-4883-a738-8a1e42576b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:40.052392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fbb78-5834-4883-a738-8a1e42576b8d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.053053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fbb78-5834-4883-a738-8a1e42576b8d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.053568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fbb78-5834-4883-a738-8a1e42576b8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.073627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fbb78-5834-4883-a738-8a1e42576b8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-d13fbb78-5834-4883-a738-8a1e42576b8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:40.084323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:40.085238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:40.086012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:40.088743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 81/157] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:40:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:40:40.094418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97428ee8-55ec-4823-b343-bf79186d5c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:40.095402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97428ee8-55ec-4823-b343-bf79186d5c43 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.095728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fbb78-5834-4883-a738-8a1e42576b8d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:40.095856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97428ee8-55ec-4823-b343-bf79186d5c43 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.096066 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97428ee8-55ec-4823-b343-bf79186d5c43 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.099993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97428ee8-55ec-4823-b343-bf79186d5c43 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:40.101713 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97428ee8-55ec-4823-b343-bf79186d5c43 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:40.103842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 77/158] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:39 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:40.110102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97428ee8-55ec-4823-b343-bf79186d5c43 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:40.297387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fbb78-5834-4883-a738-8a1e42576b8d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:40.308665 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:40.317078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:40.342619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97428ee8-55ec-4823-b343-bf79186d5c43 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:40.369933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99511803-98fa-4ca7-a2aa-adbad3e38c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:40.370660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99511803-98fa-4ca7-a2aa-adbad3e38c28 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.371457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99511803-98fa-4ca7-a2aa-adbad3e38c28 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.371760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99511803-98fa-4ca7-a2aa-adbad3e38c28 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.372499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99511803-98fa-4ca7-a2aa-adbad3e38c28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:40.372834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99511803-98fa-4ca7-a2aa-adbad3e38c28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:40.374238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99511803-98fa-4ca7-a2aa-adbad3e38c28 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:40.388691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 82/159] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:40:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:40.394273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99511803-98fa-4ca7-a2aa-adbad3e38c28 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:40.427350 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02218423-da49-422b-a9d9-ebfd8180bb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:40.428138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02218423-da49-422b-a9d9-ebfd8180bb15 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.428428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02218423-da49-422b-a9d9-ebfd8180bb15 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.428733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02218423-da49-422b-a9d9-ebfd8180bb15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.429582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02218423-da49-422b-a9d9-ebfd8180bb15 None admin] RBAC: Adding query filter params (role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be, user.id=fcdb4126379942f5816db31b2de8f861) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:40.429896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02218423-da49-422b-a9d9-ebfd8180bb15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:40.431543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02218423-da49-422b-a9d9-ebfd8180bb15 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:40.446641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 78/160] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:40 2022] GET /identity/v3/role_assignments?user.id=fcdb4126379942f5816db31b2de8f861&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:40.448178 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b1e21-2cd9-4c95-b17d-6ffcea439d4a None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.448470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b1e21-2cd9-4c95-b17d-6ffcea439d4a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.448775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b1e21-2cd9-4c95-b17d-6ffcea439d4a None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.450753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 79/161] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:40.456411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8e5c4-6f6e-4705-bfc3-1df3db615130 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.456723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8e5c4-6f6e-4705-bfc3-1df3db615130 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.457062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8e5c4-6f6e-4705-bfc3-1df3db615130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.481794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-15d8e5c4-6f6e-4705-bfc3-1df3db615130 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:40.483093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-15d8e5c4-6f6e-4705-bfc3-1df3db615130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:40.569062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99511803-98fa-4ca7-a2aa-adbad3e38c28 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:40.625492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf697ea8-cae8-4627-97d1-589f22a335dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:40.626394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf697ea8-cae8-4627-97d1-589f22a335dd None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.626676 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf697ea8-cae8-4627-97d1-589f22a335dd None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.626951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf697ea8-cae8-4627-97d1-589f22a335dd None admin] PATH_INFO: `/v3/users/fcdb4126379942f5816db31b2de8f861` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.628237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf697ea8-cae8-4627-97d1-589f22a335dd None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb4126379942f5816db31b2de8f861)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:40.629727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf697ea8-cae8-4627-97d1-589f22a335dd None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:40.631372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 83/162] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:40 2022] GET /identity/v3/users/fcdb4126379942f5816db31b2de8f861 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:40.635044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 80/163] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:40.643780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf697ea8-cae8-4627-97d1-589f22a335dd None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:40.684683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab994936-29a7-4029-8209-4c1356b0c9c9 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.685207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab994936-29a7-4029-8209-4c1356b0c9c9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.685494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab994936-29a7-4029-8209-4c1356b0c9c9 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.685937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:40.688341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 81/164] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:40.693706 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab994936-29a7-4029-8209-4c1356b0c9c9 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:40.759464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf697ea8-cae8-4627-97d1-589f22a335dd None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:40.819320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3430ab3-9112-46f7-9d51-1c5f210eb0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:40.819908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3430ab3-9112-46f7-9d51-1c5f210eb0ce None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.820187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3430ab3-9112-46f7-9d51-1c5f210eb0ce None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.820465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3430ab3-9112-46f7-9d51-1c5f210eb0ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.875093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:40.906693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab994936-29a7-4029-8209-4c1356b0c9c9 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:40.958147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439696d-1a6a-4d28-9514-737945f88f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:40.959013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439696d-1a6a-4d28-9514-737945f88f5e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.959304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439696d-1a6a-4d28-9514-737945f88f5e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.959632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439696d-1a6a-4d28-9514-737945f88f5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.978330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439696d-1a6a-4d28-9514-737945f88f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-f439696d-1a6a-4d28-9514-737945f88f5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:40.991255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:40.992168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:40.992996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 82/165] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:40:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:40.994336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506cd8c-6c1c-4fc7-a864-ba41e9be591e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:40.994427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506cd8c-6c1c-4fc7-a864-ba41e9be591e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:40.996674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506cd8c-6c1c-4fc7-a864-ba41e9be591e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:40.996803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3430ab3-9112-46f7-9d51-1c5f210eb0ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:40.997686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3430ab3-9112-46f7-9d51-1c5f210eb0ce None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:41.029217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-2506cd8c-6c1c-4fc7-a864-ba41e9be591e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:41.030446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2506cd8c-6c1c-4fc7-a864-ba41e9be591e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:41.093949 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:41.126479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 83/166] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:41.127445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506cd8c-6c1c-4fc7-a864-ba41e9be591e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:41.212087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 84/167] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:41.217498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:41.217836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3430ab3-9112-46f7-9d51-1c5f210eb0ce None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:41.266949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506cd8c-6c1c-4fc7-a864-ba41e9be591e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:41.325371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e75d3-2199-4596-97ff-d6e062b83582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:41.326074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e75d3-2199-4596-97ff-d6e062b83582 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:41.326363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e75d3-2199-4596-97ff-d6e062b83582 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:41.326639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e75d3-2199-4596-97ff-d6e062b83582 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:41.327471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e75d3-2199-4596-97ff-d6e062b83582 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:41.327782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e75d3-2199-4596-97ff-d6e062b83582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:41.329272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e75d3-2199-4596-97ff-d6e062b83582 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:41.338456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 84/168] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:41.339122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e75d3-2199-4596-97ff-d6e062b83582 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:41.395362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3430ab3-9112-46f7-9d51-1c5f210eb0ce None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:41.477967 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8db69-0706-46cf-8f43-f135c7db8110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:41.478802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8db69-0706-46cf-8f43-f135c7db8110 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:41.479088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8db69-0706-46cf-8f43-f135c7db8110 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:41.479365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8db69-0706-46cf-8f43-f135c7db8110 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:41.485333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:41.496258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8db69-0706-46cf-8f43-f135c7db8110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-06b8db69-0706-46cf-8f43-f135c7db8110 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:41.513043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:41.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:41.514775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 85/169] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:41.515406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8db69-0706-46cf-8f43-f135c7db8110 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:41.535916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e75d3-2199-4596-97ff-d6e062b83582 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:41.605095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3f6a7-5d29-4446-94e4-ada145a839b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:41.606115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3f6a7-5d29-4446-94e4-ada145a839b1 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:41.606404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3f6a7-5d29-4446-94e4-ada145a839b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:41.606681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3f6a7-5d29-4446-94e4-ada145a839b1 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:41.608013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3f6a7-5d29-4446-94e4-ada145a839b1 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:41.609517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3f6a7-5d29-4446-94e4-ada145a839b1 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:41.611543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 85/170] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:41 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:41.612332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3f6a7-5d29-4446-94e4-ada145a839b1 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:41.623361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:41.679271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8db69-0706-46cf-8f43-f135c7db8110 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:41.734709 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c33464-5520-452b-86bc-c879f71df8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:41.735617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c33464-5520-452b-86bc-c879f71df8e9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:41.735906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c33464-5520-452b-86bc-c879f71df8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:41.736187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c33464-5520-452b-86bc-c879f71df8e9 None admin] PATH_INFO: `/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:41.737450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c33464-5520-452b-86bc-c879f71df8e9 None admin] RBAC: Authorizing `identity:get_user(user_id=3cfc6b9179e849cdbe9d2d91ab4d0972)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:41.738934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c33464-5520-452b-86bc-c879f71df8e9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:41.740853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 86/171] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:41 2022] GET /identity/v3/users/3cfc6b9179e849cdbe9d2d91ab4d0972 => generated 247 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:41.741427 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c33464-5520-452b-86bc-c879f71df8e9 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:41.829306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3f6a7-5d29-4446-94e4-ada145a839b1 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:41.851307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:41.909998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450b6f2-f2f5-47df-827b-d7fa7c0bc145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:41.910790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450b6f2-f2f5-47df-827b-d7fa7c0bc145 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:41.911082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450b6f2-f2f5-47df-827b-d7fa7c0bc145 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:41.911358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450b6f2-f2f5-47df-827b-d7fa7c0bc145 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:41.912123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450b6f2-f2f5-47df-827b-d7fa7c0bc145 None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:41.912431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450b6f2-f2f5-47df-827b-d7fa7c0bc145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:41.913237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c33464-5520-452b-86bc-c879f71df8e9 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:41.913846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450b6f2-f2f5-47df-827b-d7fa7c0bc145 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:41.931680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 86/172] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:41.932256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450b6f2-f2f5-47df-827b-d7fa7c0bc145 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:41.972114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae3635-5e4c-4c72-a111-47518f0ea573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:41.972810 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3635-5e4c-4c72-a111-47518f0ea573 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:41.973099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3635-5e4c-4c72-a111-47518f0ea573 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:41.973379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3635-5e4c-4c72-a111-47518f0ea573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:41.974228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae3635-5e4c-4c72-a111-47518f0ea573 None admin] RBAC: Adding query filter params (role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be, user.id=fcdb4126379942f5816db31b2de8f861) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:41.974544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae3635-5e4c-4c72-a111-47518f0ea573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:41.975996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae3635-5e4c-4c72-a111-47518f0ea573 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:41.982565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 87/173] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:41 2022] GET /identity/v3/role_assignments?user.id=fcdb4126379942f5816db31b2de8f861&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:41.983176 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae3635-5e4c-4c72-a111-47518f0ea573 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:42.068058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:42.094783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae3635-5e4c-4c72-a111-47518f0ea573 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:42.130059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:42.138159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed707029-77ba-483f-b95d-e8254628f17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:42.142032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed707029-77ba-483f-b95d-e8254628f17e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.142032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed707029-77ba-483f-b95d-e8254628f17e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.142032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed707029-77ba-483f-b95d-e8254628f17e None admin] PATH_INFO: `/v3/users/fcdb4126379942f5816db31b2de8f861` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.142032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed707029-77ba-483f-b95d-e8254628f17e None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb4126379942f5816db31b2de8f861)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:42.142632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed707029-77ba-483f-b95d-e8254628f17e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:42.144519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 88/174] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:41 2022] GET /identity/v3/users/fcdb4126379942f5816db31b2de8f861 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:42.145659 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0ec89-8989-4b85-bad6-b5c1261dfc1f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.145951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0ec89-8989-4b85-bad6-b5c1261dfc1f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.146235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0ec89-8989-4b85-bad6-b5c1261dfc1f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.147692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 89/175] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:42.154242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb5aaf-6d46-45cf-b434-21133f6a1c7d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.154553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb5aaf-6d46-45cf-b434-21133f6a1c7d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.154966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb5aaf-6d46-45cf-b434-21133f6a1c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.156214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450b6f2-f2f5-47df-827b-d7fa7c0bc145 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:42.185380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2cbb5aaf-6d46-45cf-b434-21133f6a1c7d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:42.186527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2cbb5aaf-6d46-45cf-b434-21133f6a1c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:42.233949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a97e1-6207-46a8-b098-80364d4f82ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:42.234628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a97e1-6207-46a8-b098-80364d4f82ea None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.234915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a97e1-6207-46a8-b098-80364d4f82ea None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.235195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a97e1-6207-46a8-b098-80364d4f82ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.341334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 90/176] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:42.342683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb5aaf-6d46-45cf-b434-21133f6a1c7d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:42.359797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a97e1-6207-46a8-b098-80364d4f82ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:42.361814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a97e1-6207-46a8-b098-80364d4f82ea None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:42.445061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:42.484450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb5aaf-6d46-45cf-b434-21133f6a1c7d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:42.499943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 87/177] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:42.501708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca5035-bd21-4dbd-8da7-1317ee5ecc4e None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.502189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca5035-bd21-4dbd-8da7-1317ee5ecc4e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.502640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca5035-bd21-4dbd-8da7-1317ee5ecc4e None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.504476 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 88/178] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:42.505588 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca5035-bd21-4dbd-8da7-1317ee5ecc4e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:42.515063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:42.524117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff8f67-08c0-4c3d-baa5-dc261295f15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:42.524816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff8f67-08c0-4c3d-baa5-dc261295f15f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.525101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff8f67-08c0-4c3d-baa5-dc261295f15f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.525377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff8f67-08c0-4c3d-baa5-dc261295f15f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.599612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff8f67-08c0-4c3d-baa5-dc261295f15f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:42.601179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff8f67-08c0-4c3d-baa5-dc261295f15f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:42.620466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca5035-bd21-4dbd-8da7-1317ee5ecc4e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:42.655985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e3241-3050-4c62-a36e-63e44212d804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:42.656841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e3241-3050-4c62-a36e-63e44212d804 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.657131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e3241-3050-4c62-a36e-63e44212d804 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.657412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e3241-3050-4c62-a36e-63e44212d804 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.668390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e3241-3050-4c62-a36e-63e44212d804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-2b2e3241-3050-4c62-a36e-63e44212d804 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:42.676264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:42.677183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 89/179] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:40:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:42.685412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32452e06-3480-4c16-85ff-0ac9cfb90c6b None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.686105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32452e06-3480-4c16-85ff-0ac9cfb90c6b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.686105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32452e06-3480-4c16-85ff-0ac9cfb90c6b None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.686105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 90/180] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:42.686105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32452e06-3480-4c16-85ff-0ac9cfb90c6b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:42.712265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:42.715706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 91/181] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:42.717031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa43f-1a24-4136-9dda-11fb1a7f1931 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.717326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa43f-1a24-4136-9dda-11fb1a7f1931 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.717606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa43f-1a24-4136-9dda-11fb1a7f1931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.734533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e94fa43f-1a24-4136-9dda-11fb1a7f1931 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:42.735729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e94fa43f-1a24-4136-9dda-11fb1a7f1931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:42.758317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:42.799824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32452e06-3480-4c16-85ff-0ac9cfb90c6b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:42.824222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 92/182] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:42.824222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48d2c4-d551-435a-a11c-82aa430e5c87 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.824222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48d2c4-d551-435a-a11c-82aa430e5c87 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.824222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48d2c4-d551-435a-a11c-82aa430e5c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.843688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-6b48d2c4-d551-435a-a11c-82aa430e5c87 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:42.844945 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6b48d2c4-d551-435a-a11c-82aa430e5c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:42.847058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c1e37-db98-44ed-863a-65db74feebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:42.847937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c1e37-db98-44ed-863a-65db74feebde None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.848221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c1e37-db98-44ed-863a-65db74feebde None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.848498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c1e37-db98-44ed-863a-65db74feebde None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.849740 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c1e37-db98-44ed-863a-65db74feebde None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:42.851211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c1e37-db98-44ed-863a-65db74feebde None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:42.852971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 91/183] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:42 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:42.859086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c1e37-db98-44ed-863a-65db74feebde None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:42.917530 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 93/184] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:42.919655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48d2c4-d551-435a-a11c-82aa430e5c87 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:42.953055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:42.961673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c1e37-db98-44ed-863a-65db74feebde None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:42.996775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3163df-cf1e-4755-9f58-d5365d8a6d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:42.997453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3163df-cf1e-4755-9f58-d5365d8a6d13 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:42.997739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3163df-cf1e-4755-9f58-d5365d8a6d13 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:42.998025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3163df-cf1e-4755-9f58-d5365d8a6d13 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:42.998771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3163df-cf1e-4755-9f58-d5365d8a6d13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:42.999082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3163df-cf1e-4755-9f58-d5365d8a6d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:43.000453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3163df-cf1e-4755-9f58-d5365d8a6d13 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:43.004951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:43.008777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 92/185] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:43.009404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3163df-cf1e-4755-9f58-d5365d8a6d13 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.020063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48d2c4-d551-435a-a11c-82aa430e5c87 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.057248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63ef9c-3277-495c-9318-a1c18022dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.058119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63ef9c-3277-495c-9318-a1c18022dba2 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.058421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63ef9c-3277-495c-9318-a1c18022dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.058696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63ef9c-3277-495c-9318-a1c18022dba2 None admin] PATH_INFO: `/v3/projects/c5fbbd3f0070448e8fedfb89f9ba0e83` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.059882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63ef9c-3277-495c-9318-a1c18022dba2 None admin] RBAC: Authorizing `identity:get_project(project_id=c5fbbd3f0070448e8fedfb89f9ba0e83)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:43.061385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63ef9c-3277-495c-9318-a1c18022dba2 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:43.063106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 94/186] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:42 2022] GET /identity/v3/projects/c5fbbd3f0070448e8fedfb89f9ba0e83 => generated 312 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:43.068973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63ef9c-3277-495c-9318-a1c18022dba2 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.120089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3163df-cf1e-4755-9f58-d5365d8a6d13 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.170000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5bd25-6524-4526-af1f-f34147ef0c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.170809 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5bd25-6524-4526-af1f-f34147ef0c0e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.171097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5bd25-6524-4526-af1f-f34147ef0c0e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.171376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5bd25-6524-4526-af1f-f34147ef0c0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.172079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63ef9c-3277-495c-9318-a1c18022dba2 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.181798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5bd25-6524-4526-af1f-f34147ef0c0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-afe5bd25-6524-4526-af1f-f34147ef0c0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:43.189763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:43.190687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:43.191546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 93/187] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:43.192225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8b427-c4c3-4d46-89e7-99afbe53ab34 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.192512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8b427-c4c3-4d46-89e7-99afbe53ab34 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.192824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8b427-c4c3-4d46-89e7-99afbe53ab34 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.194118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 94/188] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:43.196828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8b427-c4c3-4d46-89e7-99afbe53ab34 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.211564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74664ee5-11d1-431f-9f65-382e174d302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.212302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74664ee5-11d1-431f-9f65-382e174d302e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.212666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74664ee5-11d1-431f-9f65-382e174d302e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.212950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74664ee5-11d1-431f-9f65-382e174d302e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.213810 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74664ee5-11d1-431f-9f65-382e174d302e None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be, user.id=3cfc6b9179e849cdbe9d2d91ab4d0972) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:43.213923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:43.214122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74664ee5-11d1-431f-9f65-382e174d302e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:43.216127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74664ee5-11d1-431f-9f65-382e174d302e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:43.222150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 95/189] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/role_assignments?user.id=3cfc6b9179e849cdbe9d2d91ab4d0972&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:43.228418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74664ee5-11d1-431f-9f65-382e174d302e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.245165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:43.323189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8b427-c4c3-4d46-89e7-99afbe53ab34 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.344136 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74664ee5-11d1-431f-9f65-382e174d302e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.369648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31bc38-16c8-4e40-92bc-ce9b62e7b016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.370763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31bc38-16c8-4e40-92bc-ce9b62e7b016 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.371613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31bc38-16c8-4e40-92bc-ce9b62e7b016 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.371956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31bc38-16c8-4e40-92bc-ce9b62e7b016 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.383187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31bc38-16c8-4e40-92bc-ce9b62e7b016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:43.383350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5a61-0254-42c0-9de9-a3d810b1e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.384289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5a61-0254-42c0-9de9-a3d810b1e142 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.384575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5a61-0254-42c0-9de9-a3d810b1e142 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.384889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5a61-0254-42c0-9de9-a3d810b1e142 None admin] PATH_INFO: `/v3/users/19266ca9549b491b88892f0b3e966217` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-5a31bc38-16c8-4e40-92bc-ce9b62e7b016 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:43.392862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:43.393761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:43.394527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 95/190] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:43.395178 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31bc38-16c8-4e40-92bc-ce9b62e7b016 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.408028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b5a61-0254-42c0-9de9-a3d810b1e142 None admin] RBAC: Authorizing `identity:get_user(user_id=19266ca9549b491b88892f0b3e966217)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:43.409586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b5a61-0254-42c0-9de9-a3d810b1e142 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:43.411381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 96/191] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/users/19266ca9549b491b88892f0b3e966217 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:43.412155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5a61-0254-42c0-9de9-a3d810b1e142 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.485002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:43.501381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:43.509355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31bc38-16c8-4e40-92bc-ce9b62e7b016 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.523978 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5a61-0254-42c0-9de9-a3d810b1e142 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.542896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9643fd-a53a-44e7-949d-bfc3400736ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.543651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9643fd-a53a-44e7-949d-bfc3400736ff None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.543939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9643fd-a53a-44e7-949d-bfc3400736ff None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.544219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9643fd-a53a-44e7-949d-bfc3400736ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.545090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9643fd-a53a-44e7-949d-bfc3400736ff None admin] RBAC: Adding query filter params (role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=c5fbbd3f0070448e8fedfb89f9ba0e83, user.id=fcdb4126379942f5816db31b2de8f861) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:43.545410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9643fd-a53a-44e7-949d-bfc3400736ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:43.546868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9643fd-a53a-44e7-949d-bfc3400736ff None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:43.552703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 96/192] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/role_assignments?user.id=fcdb4126379942f5816db31b2de8f861&scope.project.id=c5fbbd3f0070448e8fedfb89f9ba0e83&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:43.553895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289849e-fa52-490b-8d79-59273525c9af None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.554187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289849e-fa52-490b-8d79-59273525c9af None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.554472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289849e-fa52-490b-8d79-59273525c9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.566107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ddb2b-64fc-4aaf-bc2e-7946b7db85ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.567074 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ddb2b-64fc-4aaf-bc2e-7946b7db85ea None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.567358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ddb2b-64fc-4aaf-bc2e-7946b7db85ea None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.567631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ddb2b-64fc-4aaf-bc2e-7946b7db85ea None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.568411 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ddb2b-64fc-4aaf-bc2e-7946b7db85ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:43.568766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ddb2b-64fc-4aaf-bc2e-7946b7db85ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:43.570198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ddb2b-64fc-4aaf-bc2e-7946b7db85ea None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:43.581469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 97/193] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:43.584361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b289849e-fa52-490b-8d79-59273525c9af None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:43.585623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b289849e-fa52-490b-8d79-59273525c9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:43.586991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ddb2b-64fc-4aaf-bc2e-7946b7db85ea None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.668397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 97/194] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:43.669684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289849e-fa52-490b-8d79-59273525c9af None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.721509 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ddb2b-64fc-4aaf-bc2e-7946b7db85ea None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.755610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:43.767741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71862358-af0c-4b45-ad40-dc37e1e65e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.768424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71862358-af0c-4b45-ad40-dc37e1e65e0c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.769159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71862358-af0c-4b45-ad40-dc37e1e65e0c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.769159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71862358-af0c-4b45-ad40-dc37e1e65e0c None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.769778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71862358-af0c-4b45-ad40-dc37e1e65e0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:43.770087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71862358-af0c-4b45-ad40-dc37e1e65e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:43.771475 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71862358-af0c-4b45-ad40-dc37e1e65e0c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:43.777929 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:43.781715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 98/195] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:43.782656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71862358-af0c-4b45-ad40-dc37e1e65e0c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.799030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289849e-fa52-490b-8d79-59273525c9af None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.834085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132a3a0-38ad-4ad5-a437-398f13c22e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.834643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132a3a0-38ad-4ad5-a437-398f13c22e59 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.834928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132a3a0-38ad-4ad5-a437-398f13c22e59 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.835204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132a3a0-38ad-4ad5-a437-398f13c22e59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.882927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71862358-af0c-4b45-ad40-dc37e1e65e0c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:43.904662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132a3a0-38ad-4ad5-a437-398f13c22e59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:43.906190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132a3a0-38ad-4ad5-a437-398f13c22e59 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:43.917866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ae008-7ca2-429c-9944-465e88a3cd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:43.918745 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ae008-7ca2-429c-9944-465e88a3cd7e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:43.919025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ae008-7ca2-429c-9944-465e88a3cd7e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:43.919298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ae008-7ca2-429c-9944-465e88a3cd7e None admin] PATH_INFO: `/v3/users/922a719c2f784a3a8905fd5ed41c6304` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:43.920479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ae008-7ca2-429c-9944-465e88a3cd7e None admin] RBAC: Authorizing `identity:get_user(user_id=922a719c2f784a3a8905fd5ed41c6304)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:43.921937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ae008-7ca2-429c-9944-465e88a3cd7e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:43.923568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 99/196] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/users/922a719c2f784a3a8905fd5ed41c6304 => generated 289 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:43.928062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ae008-7ca2-429c-9944-465e88a3cd7e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:43.971694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:43.990340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:44.004323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 98/197] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:44.005296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132a3a0-38ad-4ad5-a437-398f13c22e59 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.023468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ae008-7ca2-429c-9944-465e88a3cd7e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:44.057708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9ae25-e83c-4626-9dc1-587dd5d31964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:44.058521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9ae25-e83c-4626-9dc1-587dd5d31964 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:44.058807 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9ae25-e83c-4626-9dc1-587dd5d31964 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:44.059082 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9ae25-e83c-4626-9dc1-587dd5d31964 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:44.060273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9ae25-e83c-4626-9dc1-587dd5d31964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:44.061786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9ae25-e83c-4626-9dc1-587dd5d31964 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:44.063446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 100/198] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:44.064238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9ae25-e83c-4626-9dc1-587dd5d31964 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.107957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132a3a0-38ad-4ad5-a437-398f13c22e59 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:44.142416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed417b-0258-4ac6-8209-b1e92ae34a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:44.143294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed417b-0258-4ac6-8209-b1e92ae34a95 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:44.143574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed417b-0258-4ac6-8209-b1e92ae34a95 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:44.143847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed417b-0258-4ac6-8209-b1e92ae34a95 None admin] PATH_INFO: `/v3/users/19266ca9549b491b88892f0b3e966217` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:44.145081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed417b-0258-4ac6-8209-b1e92ae34a95 None admin] RBAC: Authorizing `identity:get_user(user_id=19266ca9549b491b88892f0b3e966217)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:44.146521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed417b-0258-4ac6-8209-b1e92ae34a95 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:44.148163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 99/199] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:44 2022] GET /identity/v3/users/19266ca9549b491b88892f0b3e966217 => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:44.149500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed417b-0258-4ac6-8209-b1e92ae34a95 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.165580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9ae25-e83c-4626-9dc1-587dd5d31964 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:44.185123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:44.200825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d87ae-0d9a-41ce-adb2-c4f2128d13c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:44.201477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d87ae-0d9a-41ce-adb2-c4f2128d13c0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:44.201768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d87ae-0d9a-41ce-adb2-c4f2128d13c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:44.202041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d87ae-0d9a-41ce-adb2-c4f2128d13c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:44.205584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:44.253198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed417b-0258-4ac6-8209-b1e92ae34a95 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:44.272931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d87ae-0d9a-41ce-adb2-c4f2128d13c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:44.274412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d87ae-0d9a-41ce-adb2-c4f2128d13c0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:44.287649 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1c7e-98a8-4973-9a4a-f1e9e3b88adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:44.288519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1c7e-98a8-4973-9a4a-f1e9e3b88adb None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:44.288840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1c7e-98a8-4973-9a4a-f1e9e3b88adb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:44.289113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1c7e-98a8-4973-9a4a-f1e9e3b88adb None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:44.290314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1c7e-98a8-4973-9a4a-f1e9e3b88adb None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:44.292030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1c7e-98a8-4973-9a4a-f1e9e3b88adb None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:44.293772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 100/200] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:44 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:44.294529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1c7e-98a8-4973-9a4a-f1e9e3b88adb None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.373599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 101/201] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:44.374396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d87ae-0d9a-41ce-adb2-c4f2128d13c0 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.398270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1c7e-98a8-4973-9a4a-f1e9e3b88adb None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:44.400681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:44.414589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:44.433295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb44522-46e9-4bfb-90c0-9c83f0b8156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:44.434513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb44522-46e9-4bfb-90c0-9c83f0b8156b None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:44.434798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb44522-46e9-4bfb-90c0-9c83f0b8156b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:44.435075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb44522-46e9-4bfb-90c0-9c83f0b8156b None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:44.435794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb44522-46e9-4bfb-90c0-9c83f0b8156b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:44.437292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb44522-46e9-4bfb-90c0-9c83f0b8156b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:44.478695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d87ae-0d9a-41ce-adb2-c4f2128d13c0 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:44.512814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c84e-a750-4944-83fe-71c5e8c306c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:44.513474 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c84e-a750-4944-83fe-71c5e8c306c6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:44.513768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c84e-a750-4944-83fe-71c5e8c306c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:44.514044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c84e-a750-4944-83fe-71c5e8c306c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:44.524112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 101/202] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:40:44 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:44.525056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb44522-46e9-4bfb-90c0-9c83f0b8156b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.612302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716c84e-a750-4944-83fe-71c5e8c306c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:44.614173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716c84e-a750-4944-83fe-71c5e8c306c6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:44.648992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb44522-46e9-4bfb-90c0-9c83f0b8156b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:44.658809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:44.690694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2ef2e-a675-4253-a716-ae454ea2df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:44.690694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2ef2e-a675-4253-a716-ae454ea2df72 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:44.690980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2ef2e-a675-4253-a716-ae454ea2df72 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:44.691471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2ef2e-a675-4253-a716-ae454ea2df72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:44.692027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2ef2e-a675-4253-a716-ae454ea2df72 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc, user.id=19266ca9549b491b88892f0b3e966217) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:44.692397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2ef2e-a675-4253-a716-ae454ea2df72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:44.693945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2ef2e-a675-4253-a716-ae454ea2df72 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:44.701789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 102/203] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:44 2022] GET /identity/v3/role_assignments?user.id=19266ca9549b491b88892f0b3e966217&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:44.702731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2ef2e-a675-4253-a716-ae454ea2df72 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.734617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:44.741593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 102/204] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:44.747672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c84e-a750-4944-83fe-71c5e8c306c6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.860051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2ef2e-a675-4253-a716-ae454ea2df72 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:44.897748 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c84e-a750-4944-83fe-71c5e8c306c6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:44.911241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1400fae-28a8-4b69-a033-7a75e17315fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:44.912774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1400fae-28a8-4b69-a033-7a75e17315fb None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:44.913062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1400fae-28a8-4b69-a033-7a75e17315fb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:44.913338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1400fae-28a8-4b69-a033-7a75e17315fb None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:44.914668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1400fae-28a8-4b69-a033-7a75e17315fb None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:44.916170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1400fae-28a8-4b69-a033-7a75e17315fb None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:44.918841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 103/205] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:44 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:44.925619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1400fae-28a8-4b69-a033-7a75e17315fb None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.947144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1bfeb-1a3d-4bf8-bd54-390054a970c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:44.948777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1bfeb-1a3d-4bf8-bd54-390054a970c1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:44.949299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1bfeb-1a3d-4bf8-bd54-390054a970c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:44.949785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1bfeb-1a3d-4bf8-bd54-390054a970c1 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:44.951372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1bfeb-1a3d-4bf8-bd54-390054a970c1 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:44.953107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1bfeb-1a3d-4bf8-bd54-390054a970c1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:44.955284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 103/206] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:44 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:44.961478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1bfeb-1a3d-4bf8-bd54-390054a970c1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:44.968171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:45.036372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:45.048395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1400fae-28a8-4b69-a033-7a75e17315fb None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:45.088860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260cad2-7506-4a61-b891-126974ebeab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:45.089306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260cad2-7506-4a61-b891-126974ebeab9 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:45.090125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260cad2-7506-4a61-b891-126974ebeab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:45.090407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260cad2-7506-4a61-b891-126974ebeab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:45.091264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260cad2-7506-4a61-b891-126974ebeab9 None admin] RBAC: Adding query filter params (role.id=86259b0903754e10b0da0ff74bbfa7f8, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be, user.id=922a719c2f784a3a8905fd5ed41c6304) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:45.092845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260cad2-7506-4a61-b891-126974ebeab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:45.094702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260cad2-7506-4a61-b891-126974ebeab9 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:45.101535 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 104/207] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:44 2022] GET /identity/v3/role_assignments?user.id=922a719c2f784a3a8905fd5ed41c6304&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=86259b0903754e10b0da0ff74bbfa7f8 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:45.106044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1bfeb-1a3d-4bf8-bd54-390054a970c1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:45.145993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae890a6-44b9-43f5-bef9-03c9a8f46068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:45.147127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae890a6-44b9-43f5-bef9-03c9a8f46068 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:45.147635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae890a6-44b9-43f5-bef9-03c9a8f46068 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:45.148105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae890a6-44b9-43f5-bef9-03c9a8f46068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:45.149205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae890a6-44b9-43f5-bef9-03c9a8f46068 None admin] RBAC: Adding query filter params (role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc, user.id=19266ca9549b491b88892f0b3e966217) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:45.149743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae890a6-44b9-43f5-bef9-03c9a8f46068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:45.151483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae890a6-44b9-43f5-bef9-03c9a8f46068 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:45.159316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 104/208] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:44 2022] GET /identity/v3/role_assignments?user.id=19266ca9549b491b88892f0b3e966217&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:46.927179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cd718-c853-4be2-bde3-235d43a5421e None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:46.928027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cd718-c853-4be2-bde3-235d43a5421e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:46.928581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cd718-c853-4be2-bde3-235d43a5421e None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:46.930175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 105/209] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:46.935799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a809c7-1e93-492b-8070-83a2aca1e4bd None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:46.936254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a809c7-1e93-492b-8070-83a2aca1e4bd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:46.936439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a809c7-1e93-492b-8070-83a2aca1e4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:46.954527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c8a809c7-1e93-492b-8070-83a2aca1e4bd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:46.955726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c8a809c7-1e93-492b-8070-83a2aca1e4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:46.985859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d576cba-0aa6-4372-abcf-d5915186dc60 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:46.986162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d576cba-0aa6-4372-abcf-d5915186dc60 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:46.986425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d576cba-0aa6-4372-abcf-d5915186dc60 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:46.988756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 106/210] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:46.993859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef120c-b6b9-4995-9292-7b6bd95667cd None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:46.994157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef120c-b6b9-4995-9292-7b6bd95667cd None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:46.994443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef120c-b6b9-4995-9292-7b6bd95667cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.012080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e3ef120c-b6b9-4995-9292-7b6bd95667cd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:47.013326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e3ef120c-b6b9-4995-9292-7b6bd95667cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:47.029881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:47.035853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 105/211] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:47.060696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a809c7-1e93-492b-8070-83a2aca1e4bd None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.107942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 107/212] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:47.135964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766ce64-83bd-4d85-89c6-d5c46b456ee9 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.136322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766ce64-83bd-4d85-89c6-d5c46b456ee9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.137747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766ce64-83bd-4d85-89c6-d5c46b456ee9 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.139173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 108/213] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:47.141559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766ce64-83bd-4d85-89c6-d5c46b456ee9 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.177122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a809c7-1e93-492b-8070-83a2aca1e4bd None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:47.217489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7add8e-96f7-4606-a6e9-c724abd46e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:47.218335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7add8e-96f7-4606-a6e9-c724abd46e7f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.218622 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7add8e-96f7-4606-a6e9-c724abd46e7f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.218900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7add8e-96f7-4606-a6e9-c724abd46e7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.230625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7add8e-96f7-4606-a6e9-c724abd46e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:47.234654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-ae7add8e-96f7-4606-a6e9-c724abd46e7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:47.238955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:47.239811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:47.240585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 106/214] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:40:47.242006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15417ceb-0c39-48f3-90c7-2116a9bc7f23 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.242006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15417ceb-0c39-48f3-90c7-2116a9bc7f23 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.242006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15417ceb-0c39-48f3-90c7-2116a9bc7f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.258630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766ce64-83bd-4d85-89c6-d5c46b456ee9 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:47.259487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-15417ceb-0c39-48f3-90c7-2116a9bc7f23 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:47.260670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-15417ceb-0c39-48f3-90c7-2116a9bc7f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:47.295120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708ec91-0818-4211-a1e0-a616a9cbd8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:47.295927 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708ec91-0818-4211-a1e0-a616a9cbd8dd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.296215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708ec91-0818-4211-a1e0-a616a9cbd8dd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.296632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708ec91-0818-4211-a1e0-a616a9cbd8dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.307968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708ec91-0818-4211-a1e0-a616a9cbd8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-5708ec91-0818-4211-a1e0-a616a9cbd8dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:47.316424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:47.317503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:47.318386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 109/215] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:47.320469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:47.323372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708ec91-0818-4211-a1e0-a616a9cbd8dd None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.337398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 107/216] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:47.338264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15417ceb-0c39-48f3-90c7-2116a9bc7f23 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.438590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708ec91-0818-4211-a1e0-a616a9cbd8dd None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:47.443794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15417ceb-0c39-48f3-90c7-2116a9bc7f23 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:47.472330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f648976-23dd-4161-8614-ceafffeb7c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:47.472951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f648976-23dd-4161-8614-ceafffeb7c8e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.473239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f648976-23dd-4161-8614-ceafffeb7c8e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.473513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f648976-23dd-4161-8614-ceafffeb7c8e None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.474259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f648976-23dd-4161-8614-ceafffeb7c8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:47.474565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f648976-23dd-4161-8614-ceafffeb7c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:47.475929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f648976-23dd-4161-8614-ceafffeb7c8e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:47.477447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a56d1-992c-4228-a07d-25b803aa5bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:47.478417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a56d1-992c-4228-a07d-25b803aa5bda None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.478703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a56d1-992c-4228-a07d-25b803aa5bda None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.478980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a56d1-992c-4228-a07d-25b803aa5bda None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.479725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a56d1-992c-4228-a07d-25b803aa5bda None admin] RBAC: Adding query filter params (name=member) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:47.480031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a56d1-992c-4228-a07d-25b803aa5bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:47.482369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a56d1-992c-4228-a07d-25b803aa5bda None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:47.484560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 110/217] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:47.485378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f648976-23dd-4161-8614-ceafffeb7c8e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.496603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 108/218] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:47.496603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6437000-ec13-4eeb-9f99-fb27f784bd7a None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.496603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6437000-ec13-4eeb-9f99-fb27f784bd7a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.496603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6437000-ec13-4eeb-9f99-fb27f784bd7a None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.496603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 109/219] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:47.496603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6437000-ec13-4eeb-9f99-fb27f784bd7a None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.498269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:47.547335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:47.585342 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f648976-23dd-4161-8614-ceafffeb7c8e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:47.598444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6437000-ec13-4eeb-9f99-fb27f784bd7a None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:47.618528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf57dca-f198-4960-8cfe-c2c14841fb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:47.619328 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf57dca-f198-4960-8cfe-c2c14841fb0d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.619608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf57dca-f198-4960-8cfe-c2c14841fb0d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.619886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf57dca-f198-4960-8cfe-c2c14841fb0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.621635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf57dca-f198-4960-8cfe-c2c14841fb0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:47.623077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf57dca-f198-4960-8cfe-c2c14841fb0d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:47.624709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 111/220] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:47.625822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf57dca-f198-4960-8cfe-c2c14841fb0d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.634324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb8858-809e-490e-866f-b8f262cd134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:47.635190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb8858-809e-490e-866f-b8f262cd134a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.635471 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb8858-809e-490e-866f-b8f262cd134a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.635743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb8858-809e-490e-866f-b8f262cd134a None admin] PATH_INFO: `/v3/users/19266ca9549b491b88892f0b3e966217` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.636985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb8858-809e-490e-866f-b8f262cd134a None admin] RBAC: Authorizing `identity:get_user(user_id=19266ca9549b491b88892f0b3e966217)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:47.638431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb8858-809e-490e-866f-b8f262cd134a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:47.640051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 110/221] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/users/19266ca9549b491b88892f0b3e966217 => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:47.641283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1d81d-e102-4834-85ec-648e42cc61b2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.641575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1d81d-e102-4834-85ec-648e42cc61b2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.641851 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1d81d-e102-4834-85ec-648e42cc61b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.658167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-3ad1d81d-e102-4834-85ec-648e42cc61b2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:47.659365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3ad1d81d-e102-4834-85ec-648e42cc61b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:47.703654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:47.725589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf57dca-f198-4960-8cfe-c2c14841fb0d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:47.726594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 111/222] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:47.727422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1d81d-e102-4834-85ec-648e42cc61b2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.766709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7c2dc-c7b0-4c6f-8e25-20ff62902e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:47.767590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7c2dc-c7b0-4c6f-8e25-20ff62902e7f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.767876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7c2dc-c7b0-4c6f-8e25-20ff62902e7f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.768153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7c2dc-c7b0-4c6f-8e25-20ff62902e7f None admin] PATH_INFO: `/v3/users/a2ff7c84ff00498a902f9d2d76b0a425` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.779227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:47.781705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7c2dc-c7b0-4c6f-8e25-20ff62902e7f None admin] RBAC: Authorizing `identity:get_user(user_id=a2ff7c84ff00498a902f9d2d76b0a425)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:47.783164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7c2dc-c7b0-4c6f-8e25-20ff62902e7f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:47.784909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 112/223] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/users/a2ff7c84ff00498a902f9d2d76b0a425 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:47.785682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7c2dc-c7b0-4c6f-8e25-20ff62902e7f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.841801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1d81d-e102-4834-85ec-648e42cc61b2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:47.878404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63036137-e7cd-4e6e-bb5d-4072fb1e6fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:47.879075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63036137-e7cd-4e6e-bb5d-4072fb1e6fbd None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.879354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63036137-e7cd-4e6e-bb5d-4072fb1e6fbd None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.879630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63036137-e7cd-4e6e-bb5d-4072fb1e6fbd None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.880332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63036137-e7cd-4e6e-bb5d-4072fb1e6fbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:47.881839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63036137-e7cd-4e6e-bb5d-4072fb1e6fbd None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:47.885120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7c2dc-c7b0-4c6f-8e25-20ff62902e7f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:47.920604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6abf8-30cb-4223-93e1-db23d52e0586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:47.921498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6abf8-30cb-4223-93e1-db23d52e0586 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:47.921781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6abf8-30cb-4223-93e1-db23d52e0586 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:47.922056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6abf8-30cb-4223-93e1-db23d52e0586 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:47.923243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6abf8-30cb-4223-93e1-db23d52e0586 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:47.924709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6abf8-30cb-4223-93e1-db23d52e0586 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:47.926437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 113/224] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:47.927199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6abf8-30cb-4223-93e1-db23d52e0586 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:47.929666 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:47.990789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 112/225] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:40:47 2022] POST /identity/v3/users => generated 297 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:47.991913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63036137-e7cd-4e6e-bb5d-4072fb1e6fbd None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:48.032111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6abf8-30cb-4223-93e1-db23d52e0586 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:48.076095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf21bf0-e71c-41f5-82ff-77d8003cb081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:48.077162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf21bf0-e71c-41f5-82ff-77d8003cb081 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:48.077571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf21bf0-e71c-41f5-82ff-77d8003cb081 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:48.077733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf21bf0-e71c-41f5-82ff-77d8003cb081 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:48.093818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf21bf0-e71c-41f5-82ff-77d8003cb081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-caf21bf0-e71c-41f5-82ff-77d8003cb081 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:48.105002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:48.113147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 114/226] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:48.113953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf21bf0-e71c-41f5-82ff-77d8003cb081 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:48.114646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63036137-e7cd-4e6e-bb5d-4072fb1e6fbd None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:48.116469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:48.149603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff5fd6-8921-40a0-a084-e642770a5dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:48.150583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff5fd6-8921-40a0-a084-e642770a5dc6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:48.151069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff5fd6-8921-40a0-a084-e642770a5dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:48.151345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff5fd6-8921-40a0-a084-e642770a5dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:48.188855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:48.237182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf21bf0-e71c-41f5-82ff-77d8003cb081 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:48.237391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff5fd6-8921-40a0-a084-e642770a5dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:48.238475 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff5fd6-8921-40a0-a084-e642770a5dc6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:48.289600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cbab5-8e52-4a91-af4d-d4b7706aa886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:48.290443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cbab5-8e52-4a91-af4d-d4b7706aa886 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:48.290742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cbab5-8e52-4a91-af4d-d4b7706aa886 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:48.291021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cbab5-8e52-4a91-af4d-d4b7706aa886 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/users/19266ca9549b491b88892f0b3e966217/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:48.294770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cbab5-8e52-4a91-af4d-d4b7706aa886 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f531532fbb141bd81ca33ecf527a7cc, user_id=19266ca9549b491b88892f0b3e966217, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:48.296256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cbab5-8e52-4a91-af4d-d4b7706aa886 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:48.364504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 113/227] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:48.365299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff5fd6-8921-40a0-a084-e642770a5dc6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:48.370260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:48.372215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 115/228] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:40:48 2022] PUT /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/users/19266ca9549b491b88892f0b3e966217/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:48.373883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cbab5-8e52-4a91-af4d-d4b7706aa886 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:48.505020 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cbab5-8e52-4a91-af4d-d4b7706aa886 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:48.508755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff5fd6-8921-40a0-a084-e642770a5dc6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:48.546185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:48.546504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8b341-ca05-4188-b422-8e077eaf3fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:48.547194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8b341-ca05-4188-b422-8e077eaf3fef None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:48.547478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8b341-ca05-4188-b422-8e077eaf3fef None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:48.547747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8b341-ca05-4188-b422-8e077eaf3fef None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:48.548516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8b341-ca05-4188-b422-8e077eaf3fef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:48.548863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8b341-ca05-4188-b422-8e077eaf3fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:48.549482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32478228-ec69-4590-a1e7-f8479ccf3a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:48.550310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8b341-ca05-4188-b422-8e077eaf3fef None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:48.550867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32478228-ec69-4590-a1e7-f8479ccf3a22 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:48.551330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32478228-ec69-4590-a1e7-f8479ccf3a22 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:48.552117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32478228-ec69-4590-a1e7-f8479ccf3a22 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:48.554320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32478228-ec69-4590-a1e7-f8479ccf3a22 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:48.556728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32478228-ec69-4590-a1e7-f8479ccf3a22 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:48.559374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 116/229] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:48 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:48.559802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 114/230] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:40:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:48.565428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32478228-ec69-4590-a1e7-f8479ccf3a22 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:48.570214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8b341-ca05-4188-b422-8e077eaf3fef None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:48.694195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:48.721401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32478228-ec69-4590-a1e7-f8479ccf3a22 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:48.740398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8b341-ca05-4188-b422-8e077eaf3fef None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:48.777146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3bc22-a056-4a71-ac0f-813e0e617caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:48.777812 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3bc22-a056-4a71-ac0f-813e0e617caa None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:48.778094 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3bc22-a056-4a71-ac0f-813e0e617caa None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:48.778367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3bc22-a056-4a71-ac0f-813e0e617caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:48.779202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3bc22-a056-4a71-ac0f-813e0e617caa None admin] RBAC: Adding query filter params (role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc, user.id=a2ff7c84ff00498a902f9d2d76b0a425) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:48.779513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3bc22-a056-4a71-ac0f-813e0e617caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:48.782282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3bc22-a056-4a71-ac0f-813e0e617caa None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:48.792469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 117/231] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:48 2022] GET /identity/v3/role_assignments?user.id=a2ff7c84ff00498a902f9d2d76b0a425&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:48.803514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036df7c9-97a0-46ff-a182-bc1e7826cfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:48.804334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036df7c9-97a0-46ff-a182-bc1e7826cfaa None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:48.804614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036df7c9-97a0-46ff-a182-bc1e7826cfaa None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:48.804923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036df7c9-97a0-46ff-a182-bc1e7826cfaa None admin] PATH_INFO: `/v3/users/19266ca9549b491b88892f0b3e966217` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:48.806123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036df7c9-97a0-46ff-a182-bc1e7826cfaa None admin] RBAC: Authorizing `identity:get_user(user_id=19266ca9549b491b88892f0b3e966217)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:48.807649 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036df7c9-97a0-46ff-a182-bc1e7826cfaa None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:48.810001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 115/232] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:48 2022] GET /identity/v3/users/19266ca9549b491b88892f0b3e966217 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:48.815924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3bc22-a056-4a71-ac0f-813e0e617caa None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:48.895137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:48.953033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3bc22-a056-4a71-ac0f-813e0e617caa None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:49.009058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85872c0f-5a3f-4b70-820d-aba0213c2b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:49.009940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85872c0f-5a3f-4b70-820d-aba0213c2b66 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:49.010221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85872c0f-5a3f-4b70-820d-aba0213c2b66 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:49.010495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85872c0f-5a3f-4b70-820d-aba0213c2b66 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:49.011757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85872c0f-5a3f-4b70-820d-aba0213c2b66 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:49.013237 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85872c0f-5a3f-4b70-820d-aba0213c2b66 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:49.014946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 118/233] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:48 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:49.021067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036df7c9-97a0-46ff-a182-bc1e7826cfaa None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:49.148571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036df7c9-97a0-46ff-a182-bc1e7826cfaa None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:49.197923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf167aa-ec98-40bd-904e-5bb732b35286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:49.198888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf167aa-ec98-40bd-904e-5bb732b35286 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:49.199407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf167aa-ec98-40bd-904e-5bb732b35286 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:49.199692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf167aa-ec98-40bd-904e-5bb732b35286 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/users/19266ca9549b491b88892f0b3e966217/roles/31a26f4cc3674ca9bdbfc0be860a4318` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:49.205937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf167aa-ec98-40bd-904e-5bb732b35286 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f531532fbb141bd81ca33ecf527a7cc, user_id=19266ca9549b491b88892f0b3e966217, role_id=31a26f4cc3674ca9bdbfc0be860a4318)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:49.207668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf167aa-ec98-40bd-904e-5bb732b35286 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:49.211955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:49.323622 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 116/234] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:40:49 2022] PUT /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/users/19266ca9549b491b88892f0b3e966217/roles/31a26f4cc3674ca9bdbfc0be860a4318 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:50.398989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa73ac-a356-462d-a4a5-e59074f5390a None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:50.399531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa73ac-a356-462d-a4a5-e59074f5390a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:50.399531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa73ac-a356-462d-a4a5-e59074f5390a None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:50.401201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 119/235] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:50.408091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f60b5-d4bd-423b-9359-e1c5c314f63c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:50.408515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f60b5-d4bd-423b-9359-e1c5c314f63c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:50.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f60b5-d4bd-423b-9359-e1c5c314f63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:50.432298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-842f60b5-d4bd-423b-9359-e1c5c314f63c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:50.434057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-842f60b5-d4bd-423b-9359-e1c5c314f63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:50.481310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47beb20-342a-4f89-b33e-c34c8ad5c392 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:50.481958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47beb20-342a-4f89-b33e-c34c8ad5c392 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:50.482454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47beb20-342a-4f89-b33e-c34c8ad5c392 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:50.485391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 120/236] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:50.489832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701774b4-7f5a-4760-bc80-774b83f1a1f3 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:50.490350 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701774b4-7f5a-4760-bc80-774b83f1a1f3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:50.490824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701774b4-7f5a-4760-bc80-774b83f1a1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:50.525192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-701774b4-7f5a-4760-bc80-774b83f1a1f3 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:50.526713 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-701774b4-7f5a-4760-bc80-774b83f1a1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:50.530574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 117/237] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:50.567934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f60b5-d4bd-423b-9359-e1c5c314f63c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:50.622153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 121/238] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:50.687911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701774b4-7f5a-4760-bc80-774b83f1a1f3 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:50.757744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f60b5-d4bd-423b-9359-e1c5c314f63c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:50.785242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:50.806788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:50.842395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98f585-ae22-48b8-bc33-d4fea824af8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:50.843322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98f585-ae22-48b8-bc33-d4fea824af8b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:50.843609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98f585-ae22-48b8-bc33-d4fea824af8b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:50.843885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98f585-ae22-48b8-bc33-d4fea824af8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:50.849384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98f585-ae22-48b8-bc33-d4fea824af8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:50.850661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701774b4-7f5a-4760-bc80-774b83f1a1f3 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:50.850850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98f585-ae22-48b8-bc33-d4fea824af8b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:50.852674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 118/239] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:50.866097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98f585-ae22-48b8-bc33-d4fea824af8b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:50.898790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ad86-f6a8-41d6-b02c-62787d077c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:50.899967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ad86-f6a8-41d6-b02c-62787d077c56 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:50.900497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ad86-f6a8-41d6-b02c-62787d077c56 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:50.900993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ad86-f6a8-41d6-b02c-62787d077c56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:50.918704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5ad86-f6a8-41d6-b02c-62787d077c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-e8b5ad86-f6a8-41d6-b02c-62787d077c56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:50.932856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:50.934567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:50.936063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:50.945974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 122/240] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:50.949205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ad86-f6a8-41d6-b02c-62787d077c56 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:51.083161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ad86-f6a8-41d6-b02c-62787d077c56 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:51.091824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98f585-ae22-48b8-bc33-d4fea824af8b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:51.100232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:51.126187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c66565-6e5c-4597-b46f-7d273d47472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:51.127295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c66565-6e5c-4597-b46f-7d273d47472b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.127783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c66565-6e5c-4597-b46f-7d273d47472b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.128256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c66565-6e5c-4597-b46f-7d273d47472b None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.129259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c66565-6e5c-4597-b46f-7d273d47472b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:51.129778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c66565-6e5c-4597-b46f-7d273d47472b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:51.131407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c66565-6e5c-4597-b46f-7d273d47472b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:51.147582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 123/241] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:51.153963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c66565-6e5c-4597-b46f-7d273d47472b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:51.160750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19890af-40e1-446f-8b42-b8df904fe7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:51.161523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19890af-40e1-446f-8b42-b8df904fe7cf None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.161818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19890af-40e1-446f-8b42-b8df904fe7cf None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.162098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19890af-40e1-446f-8b42-b8df904fe7cf None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.162845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19890af-40e1-446f-8b42-b8df904fe7cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:51.164345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19890af-40e1-446f-8b42-b8df904fe7cf None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:51.266406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c66565-6e5c-4597-b46f-7d273d47472b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:51.304092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 119/242] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:40:50 2022] POST /identity/v3/users => generated 293 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:40:51.305885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece2829-5192-4684-97bc-aa3966134c2f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.306345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece2829-5192-4684-97bc-aa3966134c2f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.306635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c55a32-a63b-45cc-a315-fd59567b60bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:51.307074 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece2829-5192-4684-97bc-aa3966134c2f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.307870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c55a32-a63b-45cc-a315-fd59567b60bd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.307870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c55a32-a63b-45cc-a315-fd59567b60bd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.308866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c55a32-a63b-45cc-a315-fd59567b60bd None admin] PATH_INFO: `/v3/users/735e64813dcd4d5ba7823db0f8fdade9` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.310243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 120/243] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:51.316317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f4a50-423e-4ba1-a410-a20148c649e5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.316832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f4a50-423e-4ba1-a410-a20148c649e5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.316964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f4a50-423e-4ba1-a410-a20148c649e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.338498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c55a32-a63b-45cc-a315-fd59567b60bd None admin] RBAC: Authorizing `identity:get_user(user_id=735e64813dcd4d5ba7823db0f8fdade9)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:51.341177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c55a32-a63b-45cc-a315-fd59567b60bd None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:51.343667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 124/244] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:51 2022] GET /identity/v3/users/735e64813dcd4d5ba7823db0f8fdade9 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:51.349477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c55a32-a63b-45cc-a315-fd59567b60bd None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:51.369801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-958f4a50-423e-4ba1-a410-a20148c649e5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:51.371060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-958f4a50-423e-4ba1-a410-a20148c649e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:51.388409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:51.447957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:51.484675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 121/245] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:51.486796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0be923-c129-411b-a7ec-927d4cf30137 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.487289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0be923-c129-411b-a7ec-927d4cf30137 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.487948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0be923-c129-411b-a7ec-927d4cf30137 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.491153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c55a32-a63b-45cc-a315-fd59567b60bd None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:51.491897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 122/246] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:51.498546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba64e4-d07c-4568-b084-7e1fb4fff77f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.499099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba64e4-d07c-4568-b084-7e1fb4fff77f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.499536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba64e4-d07c-4568-b084-7e1fb4fff77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.518450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-71ba64e4-d07c-4568-b084-7e1fb4fff77f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:51.520055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-71ba64e4-d07c-4568-b084-7e1fb4fff77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:51.530202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977b92-59de-4d44-8446-e2c0f554e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:51.531173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977b92-59de-4d44-8446-e2c0f554e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.531291 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977b92-59de-4d44-8446-e2c0f554e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.531491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977b92-59de-4d44-8446-e2c0f554e2f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.617661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 123/247] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:51.618735 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba64e4-d07c-4568-b084-7e1fb4fff77f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:51.626965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d977b92-59de-4d44-8446-e2c0f554e2f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:51.628664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d977b92-59de-4d44-8446-e2c0f554e2f4 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:51.675566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:51.754197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba64e4-d07c-4568-b084-7e1fb4fff77f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:51.765033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:51.765226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 125/248] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:51.766058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977b92-59de-4d44-8446-e2c0f554e2f4 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:51.797150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874f05-9e27-4a17-85f6-07651e57d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:51.797150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874f05-9e27-4a17-85f6-07651e57d05f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.797150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874f05-9e27-4a17-85f6-07651e57d05f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.797150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874f05-9e27-4a17-85f6-07651e57d05f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.808995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8874f05-9e27-4a17-85f6-07651e57d05f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a8874f05-9e27-4a17-85f6-07651e57d05f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:51.818982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:51.819900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:40:51.820700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:51.823504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 124/249] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:40:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:51.824608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874f05-9e27-4a17-85f6-07651e57d05f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:51.888285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977b92-59de-4d44-8446-e2c0f554e2f4 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:51.922146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:51.927679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b318c-4a3e-4113-bba1-380fde903f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:51.928493 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b318c-4a3e-4113-bba1-380fde903f47 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.928828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b318c-4a3e-4113-bba1-380fde903f47 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.929108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b318c-4a3e-4113-bba1-380fde903f47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.938921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874f05-9e27-4a17-85f6-07651e57d05f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:51.942270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b318c-4a3e-4113-bba1-380fde903f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-760b318c-4a3e-4113-bba1-380fde903f47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:51.950843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:51.952002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:51.952877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 126/250] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:51.953518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b318c-4a3e-4113-bba1-380fde903f47 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:51.980803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7da48a-97c0-40fe-b74c-056be031a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:51.981992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7da48a-97c0-40fe-b74c-056be031a33a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:51.982476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7da48a-97c0-40fe-b74c-056be031a33a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:51.982918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7da48a-97c0-40fe-b74c-056be031a33a None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:51.984400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7da48a-97c0-40fe-b74c-056be031a33a None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:51.986152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7da48a-97c0-40fe-b74c-056be031a33a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:51.988175 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 125/251] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:51 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:51.989224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7da48a-97c0-40fe-b74c-056be031a33a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:52.049806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:52.078102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b318c-4a3e-4113-bba1-380fde903f47 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:52.129597 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee5a88-a496-4172-b8d8-6834ebb92968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:52.130275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee5a88-a496-4172-b8d8-6834ebb92968 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:52.130559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee5a88-a496-4172-b8d8-6834ebb92968 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:52.130835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee5a88-a496-4172-b8d8-6834ebb92968 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:52.131598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee5a88-a496-4172-b8d8-6834ebb92968 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:52.131907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee5a88-a496-4172-b8d8-6834ebb92968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:52.133328 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee5a88-a496-4172-b8d8-6834ebb92968 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:52.143507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 127/252] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:52.144038 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee5a88-a496-4172-b8d8-6834ebb92968 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:52.185627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7da48a-97c0-40fe-b74c-056be031a33a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:52.204956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:52.231992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef25f9-636c-4bee-94c0-2eb227cf9eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:52.232865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef25f9-636c-4bee-94c0-2eb227cf9eb0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:52.233539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef25f9-636c-4bee-94c0-2eb227cf9eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:52.233916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef25f9-636c-4bee-94c0-2eb227cf9eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:52.234771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef25f9-636c-4bee-94c0-2eb227cf9eb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:52.235203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef25f9-636c-4bee-94c0-2eb227cf9eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:52.237226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef25f9-636c-4bee-94c0-2eb227cf9eb0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:52.252037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 126/253] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:52.252037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a1290-0474-4751-a9b9-68ac448e373e None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:52.252037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a1290-0474-4751-a9b9-68ac448e373e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:52.252037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a1290-0474-4751-a9b9-68ac448e373e None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:52.252037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 127/254] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:52.256768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a1290-0474-4751-a9b9-68ac448e373e None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:52.268202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee5a88-a496-4172-b8d8-6834ebb92968 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:52.306111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e03324-261d-4e1f-9ca4-a113b3b5ab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:52.307119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e03324-261d-4e1f-9ca4-a113b3b5ab96 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:52.307623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e03324-261d-4e1f-9ca4-a113b3b5ab96 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:52.308753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e03324-261d-4e1f-9ca4-a113b3b5ab96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:52.309874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e03324-261d-4e1f-9ca4-a113b3b5ab96 None admin] RBAC: Adding query filter params (role.id=86259b0903754e10b0da0ff74bbfa7f8, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc, user.id=735e64813dcd4d5ba7823db0f8fdade9) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:52.310390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e03324-261d-4e1f-9ca4-a113b3b5ab96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:52.312438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e03324-261d-4e1f-9ca4-a113b3b5ab96 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:52.324814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 128/255] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:52 2022] GET /identity/v3/role_assignments?user.id=735e64813dcd4d5ba7823db0f8fdade9&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=86259b0903754e10b0da0ff74bbfa7f8 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:52.326259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e03324-261d-4e1f-9ca4-a113b3b5ab96 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:52.386258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:52.392563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a1290-0474-4751-a9b9-68ac448e373e None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:52.436828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5ecf2-8d2c-49c8-9610-67744749f8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:52.438026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5ecf2-8d2c-49c8-9610-67744749f8fd None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:52.438307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5ecf2-8d2c-49c8-9610-67744749f8fd None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:52.438583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5ecf2-8d2c-49c8-9610-67744749f8fd None admin] PATH_INFO: `/v3/users/19266ca9549b491b88892f0b3e966217` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:52.439877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5ecf2-8d2c-49c8-9610-67744749f8fd None admin] RBAC: Authorizing `identity:get_user(user_id=19266ca9549b491b88892f0b3e966217)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:52.441602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5ecf2-8d2c-49c8-9610-67744749f8fd None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:52.444139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 128/256] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:52 2022] GET /identity/v3/users/19266ca9549b491b88892f0b3e966217 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:52.445469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19eb15-359c-4b6f-8df9-41778e1aeb14 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:52.445768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19eb15-359c-4b6f-8df9-41778e1aeb14 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:52.446174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19eb15-359c-4b6f-8df9-41778e1aeb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:52.466287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1e19eb15-359c-4b6f-8df9-41778e1aeb14 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:52.467568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1e19eb15-359c-4b6f-8df9-41778e1aeb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:52.550950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 129/257] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:52.551932 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19eb15-359c-4b6f-8df9-41778e1aeb14 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:52.630406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e03324-261d-4e1f-9ca4-a113b3b5ab96 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:52.648738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:52.672391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0872-480d-424b-89aa-9cd364eb6c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:52.673424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0872-480d-424b-89aa-9cd364eb6c1f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:52.673785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0872-480d-424b-89aa-9cd364eb6c1f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:52.674137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0872-480d-424b-89aa-9cd364eb6c1f None admin] PATH_INFO: `/v3/users/a2ff7c84ff00498a902f9d2d76b0a425` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:52.675461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da0872-480d-424b-89aa-9cd364eb6c1f None admin] RBAC: Authorizing `identity:get_user(user_id=a2ff7c84ff00498a902f9d2d76b0a425)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:52.677119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da0872-480d-424b-89aa-9cd364eb6c1f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:52.684498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 129/258] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:52 2022] GET /identity/v3/users/a2ff7c84ff00498a902f9d2d76b0a425 => generated 297 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:52.685639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0872-480d-424b-89aa-9cd364eb6c1f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:52.694005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19eb15-359c-4b6f-8df9-41778e1aeb14 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:52.697085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:52.757685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31580c11-dd43-4e59-9c7a-dccf5abe3c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:52.758636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580c11-dd43-4e59-9c7a-dccf5abe3c8f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:52.759071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580c11-dd43-4e59-9c7a-dccf5abe3c8f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:52.759497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580c11-dd43-4e59-9c7a-dccf5abe3c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:52.834716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31580c11-dd43-4e59-9c7a-dccf5abe3c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:52.836626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31580c11-dd43-4e59-9c7a-dccf5abe3c8f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:52.838107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0872-480d-424b-89aa-9cd364eb6c1f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:52.893576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099c9b2-8eb6-4828-bb34-e55341ac30fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:52.894326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c9b2-8eb6-4828-bb34-e55341ac30fd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:52.894616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c9b2-8eb6-4828-bb34-e55341ac30fd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:52.894894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c9b2-8eb6-4828-bb34-e55341ac30fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:52.907106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099c9b2-8eb6-4828-bb34-e55341ac30fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-2099c9b2-8eb6-4828-bb34-e55341ac30fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:52.923450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:52.924385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:40:52.925277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:52.929062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 130/259] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:40:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:52.929988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099c9b2-8eb6-4828-bb34-e55341ac30fd None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:52.953671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:52.961585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 130/260] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:52.962729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31580c11-dd43-4e59-9c7a-dccf5abe3c8f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:53.017058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:53.082472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31580c11-dd43-4e59-9c7a-dccf5abe3c8f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:53.123392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099c9b2-8eb6-4828-bb34-e55341ac30fd None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:53.124445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8a1ae-0e43-4709-98b3-12ed1c273eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:53.125706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8a1ae-0e43-4709-98b3-12ed1c273eb1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:53.126294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8a1ae-0e43-4709-98b3-12ed1c273eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:53.126982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8a1ae-0e43-4709-98b3-12ed1c273eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:53.128148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8a1ae-0e43-4709-98b3-12ed1c273eb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:53.128723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8a1ae-0e43-4709-98b3-12ed1c273eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:53.130372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8a1ae-0e43-4709-98b3-12ed1c273eb1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:53.141051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 131/261] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:40:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:53.142191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8a1ae-0e43-4709-98b3-12ed1c273eb1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:53.198018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133caa-a497-4269-8d6f-aab14298408d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:53.199390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133caa-a497-4269-8d6f-aab14298408d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:53.199676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133caa-a497-4269-8d6f-aab14298408d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:53.199957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133caa-a497-4269-8d6f-aab14298408d None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:53.201330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05133caa-a497-4269-8d6f-aab14298408d None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:53.202925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05133caa-a497-4269-8d6f-aab14298408d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:53.213183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 131/262] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:52 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:53.213921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133caa-a497-4269-8d6f-aab14298408d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:53.215412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:53.252575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8a1ae-0e43-4709-98b3-12ed1c273eb1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:53.292964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc3b58-b202-4e25-9dfd-0902100a0dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:53.294414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc3b58-b202-4e25-9dfd-0902100a0dfb None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:53.294904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc3b58-b202-4e25-9dfd-0902100a0dfb None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:53.296134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc3b58-b202-4e25-9dfd-0902100a0dfb None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:53.298393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc3b58-b202-4e25-9dfd-0902100a0dfb None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:53.300281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc3b58-b202-4e25-9dfd-0902100a0dfb None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:53.303744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 132/263] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:53 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:53.305696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc3b58-b202-4e25-9dfd-0902100a0dfb None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:53.434957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc3b58-b202-4e25-9dfd-0902100a0dfb None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:53.458609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133caa-a497-4269-8d6f-aab14298408d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:53.479397 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:53.488474 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5658538-a305-4c47-a1fb-83703a11c251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:53.489094 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:53.489419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5658538-a305-4c47-a1fb-83703a11c251 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:53.489709 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5658538-a305-4c47-a1fb-83703a11c251 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:53.490044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5658538-a305-4c47-a1fb-83703a11c251 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/users/a2ff7c84ff00498a902f9d2d76b0a425/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:53.493113 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5658538-a305-4c47-a1fb-83703a11c251 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f531532fbb141bd81ca33ecf527a7cc, user_id=a2ff7c84ff00498a902f9d2d76b0a425, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:53.494605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5658538-a305-4c47-a1fb-83703a11c251 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:53.522517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddfc1e-417f-4305-b4a3-dc54ed6deb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:53.524063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddfc1e-417f-4305-b4a3-dc54ed6deb33 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:53.524377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddfc1e-417f-4305-b4a3-dc54ed6deb33 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:53.524723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddfc1e-417f-4305-b4a3-dc54ed6deb33 None admin] PATH_INFO: `/v3/users/19266ca9549b491b88892f0b3e966217` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:53.526123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddfc1e-417f-4305-b4a3-dc54ed6deb33 None admin] RBAC: Authorizing `identity:get_user(user_id=19266ca9549b491b88892f0b3e966217)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:53.527757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddfc1e-417f-4305-b4a3-dc54ed6deb33 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:53.530033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 132/264] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:53 2022] GET /identity/v3/users/19266ca9549b491b88892f0b3e966217 => generated 283 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:53.530887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddfc1e-417f-4305-b4a3-dc54ed6deb33 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:53.579804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 133/265] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:40:53 2022] PUT /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/users/a2ff7c84ff00498a902f9d2d76b0a425/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:53.580687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5658538-a305-4c47-a1fb-83703a11c251 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:53.713929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddfc1e-417f-4305-b4a3-dc54ed6deb33 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:53.773620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5658538-a305-4c47-a1fb-83703a11c251 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:53.799803 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785a1f6-4444-497f-b0e9-bf9097d98360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:53.804678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785a1f6-4444-497f-b0e9-bf9097d98360 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:53.804678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785a1f6-4444-497f-b0e9-bf9097d98360 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:53.804678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785a1f6-4444-497f-b0e9-bf9097d98360 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:53.804678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785a1f6-4444-497f-b0e9-bf9097d98360 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc, user.id=19266ca9549b491b88892f0b3e966217) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:53.804678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785a1f6-4444-497f-b0e9-bf9097d98360 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:53.804678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785a1f6-4444-497f-b0e9-bf9097d98360 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:53.822822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 133/266] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:53 2022] GET /identity/v3/role_assignments?user.id=19266ca9549b491b88892f0b3e966217&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:53.835832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af26a8-7070-4b7c-81b2-7b2d95c26ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:53.836934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af26a8-7070-4b7c-81b2-7b2d95c26ca6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:53.837452 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af26a8-7070-4b7c-81b2-7b2d95c26ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:53.837930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af26a8-7070-4b7c-81b2-7b2d95c26ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:53.909096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:53.927396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af26a8-7070-4b7c-81b2-7b2d95c26ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:53.930014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af26a8-7070-4b7c-81b2-7b2d95c26ca6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:54.067751 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 134/267] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:54.073704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785a1f6-4444-497f-b0e9-bf9097d98360 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:54.115046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cba68-13ae-46d5-b7f9-2cf3b255b658 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:54.115809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cba68-13ae-46d5-b7f9-2cf3b255b658 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:54.117414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cba68-13ae-46d5-b7f9-2cf3b255b658 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:54.119184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 135/268] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:54.124871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2db9a-4c69-4bd9-bcc7-12aa0bf7a260 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:54.125436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2db9a-4c69-4bd9-bcc7-12aa0bf7a260 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:54.125960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2db9a-4c69-4bd9-bcc7-12aa0bf7a260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:54.157687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-93c2db9a-4c69-4bd9-bcc7-12aa0bf7a260 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:54.159224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-93c2db9a-4c69-4bd9-bcc7-12aa0bf7a260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:54.163991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:54.272156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 136/269] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:54.280172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785a1f6-4444-497f-b0e9-bf9097d98360 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:54.315328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2db9a-4c69-4bd9-bcc7-12aa0bf7a260 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:54.330878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a8054-b097-4e1c-a915-0c8c170230f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:54.332828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8054-b097-4e1c-a915-0c8c170230f9 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:54.333494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8054-b097-4e1c-a915-0c8c170230f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:54.335422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8054-b097-4e1c-a915-0c8c170230f9 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:54.337882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a8054-b097-4e1c-a915-0c8c170230f9 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:54.340365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a8054-b097-4e1c-a915-0c8c170230f9 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:54.345196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:54.347255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 134/270] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:54 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:54.351313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a8054-b097-4e1c-a915-0c8c170230f9 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:54.486437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a8054-b097-4e1c-a915-0c8c170230f9 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:54.502186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2db9a-4c69-4bd9-bcc7-12aa0bf7a260 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:54.504589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:54.528683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46e076-08bf-4724-94c3-0c353c29ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:54.529539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46e076-08bf-4724-94c3-0c353c29ac68 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:54.529923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46e076-08bf-4724-94c3-0c353c29ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:54.530299 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46e076-08bf-4724-94c3-0c353c29ac68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:54.531240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46e076-08bf-4724-94c3-0c353c29ac68 None admin] RBAC: Adding query filter params (role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc, user.id=19266ca9549b491b88892f0b3e966217) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:54.531652 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46e076-08bf-4724-94c3-0c353c29ac68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:54.533337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46e076-08bf-4724-94c3-0c353c29ac68 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:54.548671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 135/271] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:54 2022] GET /identity/v3/role_assignments?user.id=19266ca9549b491b88892f0b3e966217&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:54.548671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb63c9-1b2b-4ce8-b20c-561ce707a151 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:54.548671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb63c9-1b2b-4ce8-b20c-561ce707a151 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:54.548671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb63c9-1b2b-4ce8-b20c-561ce707a151 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:54.548671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 136/272] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:54.556292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbe652-e1fc-47c8-98bd-6f35c39009b4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:54.556586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbe652-e1fc-47c8-98bd-6f35c39009b4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:54.556902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbe652-e1fc-47c8-98bd-6f35c39009b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:54.566993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98180ffa-d8bb-41ed-ac65-bac54c4cda6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:54.567608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98180ffa-d8bb-41ed-ac65-bac54c4cda6a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:54.567894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98180ffa-d8bb-41ed-ac65-bac54c4cda6a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:54.571418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98180ffa-d8bb-41ed-ac65-bac54c4cda6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:54.585344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-bddbe652-e1fc-47c8-98bd-6f35c39009b4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:54.586511 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-bddbe652-e1fc-47c8-98bd-6f35c39009b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:54.607142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98180ffa-d8bb-41ed-ac65-bac54c4cda6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-98180ffa-d8bb-41ed-ac65-bac54c4cda6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:54.615722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:54.616700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:54.617483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:54.619475 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 137/273] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:54.629999 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98180ffa-d8bb-41ed-ac65-bac54c4cda6a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:54.756669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 137/274] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:54.776942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:54.781742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbe652-e1fc-47c8-98bd-6f35c39009b4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:54.814214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98180ffa-d8bb-41ed-ac65-bac54c4cda6a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:54.868057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdcc24-5054-4572-a421-7840b179c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:54.868677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdcc24-5054-4572-a421-7840b179c226 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:54.868950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdcc24-5054-4572-a421-7840b179c226 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:54.869228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdcc24-5054-4572-a421-7840b179c226 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:54.870005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bdcc24-5054-4572-a421-7840b179c226 None admin] RBAC: Adding query filter params (name=member) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:54.870314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bdcc24-5054-4572-a421-7840b179c226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:54.872268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bdcc24-5054-4572-a421-7840b179c226 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:54.882522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 138/275] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:54.890542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdcc24-5054-4572-a421-7840b179c226 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:54.927864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbe652-e1fc-47c8-98bd-6f35c39009b4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:54.941367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:54.977717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f23bfd-2fbd-4892-b0d9-82c8f49c0697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:54.978890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f23bfd-2fbd-4892-b0d9-82c8f49c0697 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:54.979379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f23bfd-2fbd-4892-b0d9-82c8f49c0697 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:54.979857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f23bfd-2fbd-4892-b0d9-82c8f49c0697 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:54.981906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f23bfd-2fbd-4892-b0d9-82c8f49c0697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:54.984270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f23bfd-2fbd-4892-b0d9-82c8f49c0697 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:54.987494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 138/276] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:54.993526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f23bfd-2fbd-4892-b0d9-82c8f49c0697 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:55.027518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdcc24-5054-4572-a421-7840b179c226 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:55.076597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba70e10-af45-4915-981d-e4de8e8a5c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:55.077892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba70e10-af45-4915-981d-e4de8e8a5c4f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:55.078179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba70e10-af45-4915-981d-e4de8e8a5c4f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:55.078456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba70e10-af45-4915-981d-e4de8e8a5c4f None admin] PATH_INFO: `/v3/users/342a844efb1948128076c24c198c9d89` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:55.093277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:55.102652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba70e10-af45-4915-981d-e4de8e8a5c4f None admin] RBAC: Authorizing `identity:get_user(user_id=342a844efb1948128076c24c198c9d89)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:55.104156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba70e10-af45-4915-981d-e4de8e8a5c4f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:55.107129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 139/277] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:54 2022] GET /identity/v3/users/342a844efb1948128076c24c198c9d89 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:55.113688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba70e10-af45-4915-981d-e4de8e8a5c4f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:55.153855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f23bfd-2fbd-4892-b0d9-82c8f49c0697 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:55.205918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89449bbd-e13a-45bd-bb59-c3b202db8c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:55.206904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89449bbd-e13a-45bd-bb59-c3b202db8c0c None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:55.207351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89449bbd-e13a-45bd-bb59-c3b202db8c0c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:55.207632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89449bbd-e13a-45bd-bb59-c3b202db8c0c None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:55.208360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89449bbd-e13a-45bd-bb59-c3b202db8c0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:55.209852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89449bbd-e13a-45bd-bb59-c3b202db8c0c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:55.231359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba70e10-af45-4915-981d-e4de8e8a5c4f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:55.265764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e67cbc-74e0-46b3-9dd1-50bd49f93813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:55.266423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e67cbc-74e0-46b3-9dd1-50bd49f93813 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:55.266706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e67cbc-74e0-46b3-9dd1-50bd49f93813 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:55.266978 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e67cbc-74e0-46b3-9dd1-50bd49f93813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:55.267802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e67cbc-74e0-46b3-9dd1-50bd49f93813 None admin] RBAC: Adding query filter params (role.id=2a20315693ce44d696f86173c0f6eb09, user.id=342a844efb1948128076c24c198c9d89, scope.system=all) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:55.268112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e67cbc-74e0-46b3-9dd1-50bd49f93813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:55.269633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e67cbc-74e0-46b3-9dd1-50bd49f93813 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:55.275020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 140/278] 10.222.0.17 () {56 vars in 1308 bytes} [Sun Jul 24 19:40:55 2022] GET /identity/v3/role_assignments?user.id=342a844efb1948128076c24c198c9d89&scope.system=all&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:55.276399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ee04a-befa-4764-9521-2072202dc128 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:55.276740 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ee04a-befa-4764-9521-2072202dc128 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:55.277039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ee04a-befa-4764-9521-2072202dc128 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:55.280962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 141/279] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:55.283903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2a272-697d-41df-8f8e-16199778c081 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:55.289138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2a272-697d-41df-8f8e-16199778c081 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:55.289138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2a272-697d-41df-8f8e-16199778c081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:55.303634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-72e2a272-697d-41df-8f8e-16199778c081 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:55.304912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-72e2a272-697d-41df-8f8e-16199778c081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:55.316956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 139/280] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:40:54 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:40:55.382958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:55.396943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 142/281] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:55.423062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89449bbd-e13a-45bd-bb59-c3b202db8c0c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:55.469790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:55.562401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89449bbd-e13a-45bd-bb59-c3b202db8c0c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:55.602473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382f977-507b-45f6-83f8-8b230fd52890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:55.603297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382f977-507b-45f6-83f8-8b230fd52890 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:55.603580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382f977-507b-45f6-83f8-8b230fd52890 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:55.603901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382f977-507b-45f6-83f8-8b230fd52890 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:55.616629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382f977-507b-45f6-83f8-8b230fd52890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-b382f977-507b-45f6-83f8-8b230fd52890 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:55.624660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:55.625621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:55.626412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 140/282] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:55.632091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2a272-697d-41df-8f8e-16199778c081 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:55.758640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2a272-697d-41df-8f8e-16199778c081 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:55.800382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6f27c-7538-4e4e-94f9-1002b9cb35bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:55.801144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6f27c-7538-4e4e-94f9-1002b9cb35bd None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:55.801427 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6f27c-7538-4e4e-94f9-1002b9cb35bd None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:55.801703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6f27c-7538-4e4e-94f9-1002b9cb35bd None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:55.802464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6f27c-7538-4e4e-94f9-1002b9cb35bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:55.802770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6f27c-7538-4e4e-94f9-1002b9cb35bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:55.804210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6f27c-7538-4e4e-94f9-1002b9cb35bd None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:55.813661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 143/283] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:55.819225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382f977-507b-45f6-83f8-8b230fd52890 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:55.920315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:55.929550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382f977-507b-45f6-83f8-8b230fd52890 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:55.966015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d55ef-46ed-4a80-bd3b-41621d9cd8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:55.966904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d55ef-46ed-4a80-bd3b-41621d9cd8f5 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:55.967186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d55ef-46ed-4a80-bd3b-41621d9cd8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:55.967456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d55ef-46ed-4a80-bd3b-41621d9cd8f5 None admin] PATH_INFO: `/v3/users/735e64813dcd4d5ba7823db0f8fdade9` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:55.968819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d55ef-46ed-4a80-bd3b-41621d9cd8f5 None admin] RBAC: Authorizing `identity:get_user(user_id=735e64813dcd4d5ba7823db0f8fdade9)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:55.970296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d55ef-46ed-4a80-bd3b-41621d9cd8f5 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:55.972251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 141/284] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:55 2022] GET /identity/v3/users/735e64813dcd4d5ba7823db0f8fdade9 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:55.977859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6f27c-7538-4e4e-94f9-1002b9cb35bd None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:56.047270 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:56.103604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6f27c-7538-4e4e-94f9-1002b9cb35bd None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:56.146358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61c7e4-4449-4419-bf1b-c98f81d6c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:56.147235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61c7e4-4449-4419-bf1b-c98f81d6c316 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:56.147518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61c7e4-4449-4419-bf1b-c98f81d6c316 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:56.147792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61c7e4-4449-4419-bf1b-c98f81d6c316 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:56.151215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61c7e4-4449-4419-bf1b-c98f81d6c316 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:56.152769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61c7e4-4449-4419-bf1b-c98f81d6c316 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:56.155031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 144/285] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:55 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:56.160871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d55ef-46ed-4a80-bd3b-41621d9cd8f5 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:56.162508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2a2c4-90c3-417b-bba6-cf2b9a81786c None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:56.162808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2a2c4-90c3-417b-bba6-cf2b9a81786c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:56.163083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2a2c4-90c3-417b-bba6-cf2b9a81786c None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:56.164490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 145/286] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:56.170365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ede5f-34b2-4d08-9d3c-bcb332afa301 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:56.170654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ede5f-34b2-4d08-9d3c-bcb332afa301 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:56.170933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ede5f-34b2-4d08-9d3c-bcb332afa301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:56.197882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a90ede5f-34b2-4d08-9d3c-bcb332afa301 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:56.199896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a90ede5f-34b2-4d08-9d3c-bcb332afa301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:56.282702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d55ef-46ed-4a80-bd3b-41621d9cd8f5 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:56.323514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6ef38-e65d-4dc7-b6bb-3830f1afc2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:56.323514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6ef38-e65d-4dc7-b6bb-3830f1afc2b2 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:56.323514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6ef38-e65d-4dc7-b6bb-3830f1afc2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:56.323514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6ef38-e65d-4dc7-b6bb-3830f1afc2b2 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/users/735e64813dcd4d5ba7823db0f8fdade9/roles/86259b0903754e10b0da0ff74bbfa7f8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:56.326041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6ef38-e65d-4dc7-b6bb-3830f1afc2b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f531532fbb141bd81ca33ecf527a7cc, user_id=735e64813dcd4d5ba7823db0f8fdade9, role_id=86259b0903754e10b0da0ff74bbfa7f8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:56.327603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6ef38-e65d-4dc7-b6bb-3830f1afc2b2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:56.339609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 146/287] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:56.363194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ede5f-34b2-4d08-9d3c-bcb332afa301 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:56.408125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 142/288] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:40:56 2022] PUT /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/users/735e64813dcd4d5ba7823db0f8fdade9/roles/86259b0903754e10b0da0ff74bbfa7f8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:56.445158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:56.507995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ede5f-34b2-4d08-9d3c-bcb332afa301 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:56.563487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4645a6-a99f-4ebc-9e73-fed22eb7d17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:56.564327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4645a6-a99f-4ebc-9e73-fed22eb7d17f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:56.564612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4645a6-a99f-4ebc-9e73-fed22eb7d17f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:56.564912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4645a6-a99f-4ebc-9e73-fed22eb7d17f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:56.581542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4645a6-a99f-4ebc-9e73-fed22eb7d17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-cb4645a6-a99f-4ebc-9e73-fed22eb7d17f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:56.589572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:56.590533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:56.592078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 147/289] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:40:56.599167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6ef38-e65d-4dc7-b6bb-3830f1afc2b2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:56.632664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:56.722916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6ef38-e65d-4dc7-b6bb-3830f1afc2b2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:56.760534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f7692-ec3e-4197-8251-934aff957dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:56.761423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f7692-ec3e-4197-8251-934aff957dc1 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:56.761717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f7692-ec3e-4197-8251-934aff957dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:56.762000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f7692-ec3e-4197-8251-934aff957dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:56.762776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f7692-ec3e-4197-8251-934aff957dc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:56.763092 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f7692-ec3e-4197-8251-934aff957dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:56.764531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f7692-ec3e-4197-8251-934aff957dc1 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:56.773972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 143/290] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:56.781776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4645a6-a99f-4ebc-9e73-fed22eb7d17f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:56.912066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4645a6-a99f-4ebc-9e73-fed22eb7d17f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:56.924164 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:56.947358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55df4-0a7c-43cb-8e1d-5def2cbecb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:56.948239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55df4-0a7c-43cb-8e1d-5def2cbecb34 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:56.948572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55df4-0a7c-43cb-8e1d-5def2cbecb34 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:56.948855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55df4-0a7c-43cb-8e1d-5def2cbecb34 None admin] PATH_INFO: `/v3/users/a2ff7c84ff00498a902f9d2d76b0a425` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:56.955013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55df4-0a7c-43cb-8e1d-5def2cbecb34 None admin] RBAC: Authorizing `identity:get_user(user_id=a2ff7c84ff00498a902f9d2d76b0a425)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:56.955013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55df4-0a7c-43cb-8e1d-5def2cbecb34 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:56.955013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 148/291] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:56 2022] GET /identity/v3/users/a2ff7c84ff00498a902f9d2d76b0a425 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:56.958781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f7692-ec3e-4197-8251-934aff957dc1 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:57.075567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:57.098860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f7692-ec3e-4197-8251-934aff957dc1 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:57.143525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd72d-189f-47e9-9c9a-cd8a95c7546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:57.144183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd72d-189f-47e9-9c9a-cd8a95c7546b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:57.144467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd72d-189f-47e9-9c9a-cd8a95c7546b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:57.144762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd72d-189f-47e9-9c9a-cd8a95c7546b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:57.238446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd72d-189f-47e9-9c9a-cd8a95c7546b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:57.240298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd72d-189f-47e9-9c9a-cd8a95c7546b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:57.343338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:57.367304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 144/292] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:40:57.375782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55df4-0a7c-43cb-8e1d-5def2cbecb34 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:57.544171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55df4-0a7c-43cb-8e1d-5def2cbecb34 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:57.589020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d462e27-5b78-440c-859e-a64b3f7b7297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:57.590485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d462e27-5b78-440c-859e-a64b3f7b7297 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:57.590767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d462e27-5b78-440c-859e-a64b3f7b7297 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:57.591041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d462e27-5b78-440c-859e-a64b3f7b7297 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:57.597388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d462e27-5b78-440c-859e-a64b3f7b7297 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:57.598837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d462e27-5b78-440c-859e-a64b3f7b7297 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:57.600933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 149/293] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:40:57 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:57.606691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd72d-189f-47e9-9c9a-cd8a95c7546b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:57.644763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fd2ee-4ac9-4ea9-a3d1-7b6be050bcbc None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:57.645076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fd2ee-4ac9-4ea9-a3d1-7b6be050bcbc None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:57.645318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fd2ee-4ac9-4ea9-a3d1-7b6be050bcbc None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:57.646684 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 150/294] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:57.653536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8362adf-3d3b-46ec-bee7-d85ac201cd0f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:57.653832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8362adf-3d3b-46ec-bee7-d85ac201cd0f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:57.654152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8362adf-3d3b-46ec-bee7-d85ac201cd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:57.685738 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-b8362adf-3d3b-46ec-bee7-d85ac201cd0f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:57.686961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b8362adf-3d3b-46ec-bee7-d85ac201cd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:57.737609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd72d-189f-47e9-9c9a-cd8a95c7546b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:57.775250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be976316-26ff-4508-a578-c5bf1e4134d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:57.775929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be976316-26ff-4508-a578-c5bf1e4134d0 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:57.776218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be976316-26ff-4508-a578-c5bf1e4134d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:57.776505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be976316-26ff-4508-a578-c5bf1e4134d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:57.784838 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be976316-26ff-4508-a578-c5bf1e4134d0 None admin] RBAC: Adding query filter params (role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc, user.id=a2ff7c84ff00498a902f9d2d76b0a425) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:57.784838 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be976316-26ff-4508-a578-c5bf1e4134d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:57.784838 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be976316-26ff-4508-a578-c5bf1e4134d0 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:57.786332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 145/295] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:40:57 2022] GET /identity/v3/role_assignments?user.id=a2ff7c84ff00498a902f9d2d76b0a425&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:57.803790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:57.834088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 151/296] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:40:57.869514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be976316-26ff-4508-a578-c5bf1e4134d0 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:57.944499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:58.014442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be976316-26ff-4508-a578-c5bf1e4134d0 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:58.051389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b71951-204b-439b-b03c-5a2103ee1502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:58.052217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b71951-204b-439b-b03c-5a2103ee1502 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.052507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b71951-204b-439b-b03c-5a2103ee1502 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.052858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b71951-204b-439b-b03c-5a2103ee1502 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.064686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b71951-204b-439b-b03c-5a2103ee1502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-54b71951-204b-439b-b03c-5a2103ee1502 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:58.072720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:58.083447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:40:58.084231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 146/297] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:58.087371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8362adf-3d3b-46ec-bee7-d85ac201cd0f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:58.223521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8362adf-3d3b-46ec-bee7-d85ac201cd0f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:58.265276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c9e26-75d3-4a5e-af15-69a1139d3ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:58.266328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c9e26-75d3-4a5e-af15-69a1139d3ec6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.266869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c9e26-75d3-4a5e-af15-69a1139d3ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.267357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c9e26-75d3-4a5e-af15-69a1139d3ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.268359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c9e26-75d3-4a5e-af15-69a1139d3ec6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:58.268967 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c9e26-75d3-4a5e-af15-69a1139d3ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:58.270578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c9e26-75d3-4a5e-af15-69a1139d3ec6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:58.280412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 152/298] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:58.286386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b71951-204b-439b-b03c-5a2103ee1502 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:58.292911 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8673-0e01-47e9-a402-035a998a38fc None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.293468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8673-0e01-47e9-a402-035a998a38fc None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.293964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8673-0e01-47e9-a402-035a998a38fc None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.295761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 153/299] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:40:58.305707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443c119-cf0e-42be-b2fe-50f97f0c2e5f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.306059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443c119-cf0e-42be-b2fe-50f97f0c2e5f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.307540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443c119-cf0e-42be-b2fe-50f97f0c2e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.332125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-3443c119-cf0e-42be-b2fe-50f97f0c2e5f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:58.333406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3443c119-cf0e-42be-b2fe-50f97f0c2e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:58.365979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:58.409268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 154/300] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:58.426273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b71951-204b-439b-b03c-5a2103ee1502 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:58.439155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:58.443617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443c119-cf0e-42be-b2fe-50f97f0c2e5f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:58.478349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05417de6-4476-400c-bb87-806d87030c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:58.479312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05417de6-4476-400c-bb87-806d87030c8d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.479610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05417de6-4476-400c-bb87-806d87030c8d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.480730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05417de6-4476-400c-bb87-806d87030c8d None admin] PATH_INFO: `/v3/users/342a844efb1948128076c24c198c9d89` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.483340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05417de6-4476-400c-bb87-806d87030c8d None admin] RBAC: Authorizing `identity:get_user(user_id=342a844efb1948128076c24c198c9d89)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:58.485273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05417de6-4476-400c-bb87-806d87030c8d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:58.487620 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 147/301] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:58 2022] GET /identity/v3/users/342a844efb1948128076c24c198c9d89 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:58.496039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05417de6-4476-400c-bb87-806d87030c8d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:58.586190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443c119-cf0e-42be-b2fe-50f97f0c2e5f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:58.624606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05417de6-4476-400c-bb87-806d87030c8d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:58.626856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2097f-c80a-4a63-902d-2da78a99f6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:58.629635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2097f-c80a-4a63-902d-2da78a99f6dd None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.629635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2097f-c80a-4a63-902d-2da78a99f6dd None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.629635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2097f-c80a-4a63-902d-2da78a99f6dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.639844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2097f-c80a-4a63-902d-2da78a99f6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-b1c2097f-c80a-4a63-902d-2da78a99f6dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:58.647712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:58.648853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:58.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:58.654261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 155/302] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:40:58.655709 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622e6b8-2f54-40bd-9407-cd49532eb72a None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.656010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622e6b8-2f54-40bd-9407-cd49532eb72a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.656299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622e6b8-2f54-40bd-9407-cd49532eb72a None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.657976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 156/303] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:58.658725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622e6b8-2f54-40bd-9407-cd49532eb72a None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:58.673513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017f66e-2c93-4852-ab27-2f2c15dad917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:58.674567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017f66e-2c93-4852-ab27-2f2c15dad917 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.674862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017f66e-2c93-4852-ab27-2f2c15dad917 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.675148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017f66e-2c93-4852-ab27-2f2c15dad917 None admin] PATH_INFO: `/v3/system/users/342a844efb1948128076c24c198c9d89/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.676729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017f66e-2c93-4852-ab27-2f2c15dad917 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=342a844efb1948128076c24c198c9d89, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:58.678219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017f66e-2c93-4852-ab27-2f2c15dad917 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:58.691211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:58.773812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622e6b8-2f54-40bd-9407-cd49532eb72a None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:58.774769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 148/304] 10.222.0.17 () {58 vars in 1334 bytes} [Sun Jul 24 19:40:58 2022] PUT /identity/v3/system/users/342a844efb1948128076c24c198c9d89/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:40:58.776499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1aeb5-361c-4eff-89ec-15c642c57dcf None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.777701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1aeb5-361c-4eff-89ec-15c642c57dcf None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.777867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1aeb5-361c-4eff-89ec-15c642c57dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.812064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-2dc1aeb5-361c-4eff-89ec-15c642c57dcf None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:58.815692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2dc1aeb5-361c-4eff-89ec-15c642c57dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:58.825385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86962be4-5ce5-425d-ad09-fce2d1eb87b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:58.827565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86962be4-5ce5-425d-ad09-fce2d1eb87b7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:58.828085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86962be4-5ce5-425d-ad09-fce2d1eb87b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:58.828617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86962be4-5ce5-425d-ad09-fce2d1eb87b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:58.829655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86962be4-5ce5-425d-ad09-fce2d1eb87b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:58.830957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86962be4-5ce5-425d-ad09-fce2d1eb87b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:58.832763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86962be4-5ce5-425d-ad09-fce2d1eb87b7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:58.852492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 157/305] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:58.855884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86962be4-5ce5-425d-ad09-fce2d1eb87b7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:58.869145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:58.932714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 149/306] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:58.958151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1aeb5-361c-4eff-89ec-15c642c57dcf None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:58.987377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:59.002405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86962be4-5ce5-425d-ad09-fce2d1eb87b7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:59.048532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503908e-ec00-4222-be70-8b7f08d6ebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:59.049479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503908e-ec00-4222-be70-8b7f08d6ebf8 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:59.049764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503908e-ec00-4222-be70-8b7f08d6ebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:59.050040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503908e-ec00-4222-be70-8b7f08d6ebf8 None admin] PATH_INFO: `/v3/users/50c39f084928467b9e0bcf09edf9941e` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:59.065976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503908e-ec00-4222-be70-8b7f08d6ebf8 None admin] RBAC: Authorizing `identity:get_user(user_id=50c39f084928467b9e0bcf09edf9941e)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:59.067459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503908e-ec00-4222-be70-8b7f08d6ebf8 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:59.069888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 158/307] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:58 2022] GET /identity/v3/users/50c39f084928467b9e0bcf09edf9941e => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:59.071019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8b69f-2b1b-4fde-beb2-f470a899ae9b None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:59.071307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8b69f-2b1b-4fde-beb2-f470a899ae9b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:59.071593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8b69f-2b1b-4fde-beb2-f470a899ae9b None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:59.076977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 159/308] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:40:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:40:59.081599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8b69f-2b1b-4fde-beb2-f470a899ae9b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:59.081925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1aeb5-361c-4eff-89ec-15c642c57dcf None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:59.125874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4700d-74d9-4f76-8c00-ce5f703f1955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:59.126600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4700d-74d9-4f76-8c00-ce5f703f1955 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:59.126982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4700d-74d9-4f76-8c00-ce5f703f1955 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:59.127170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4700d-74d9-4f76-8c00-ce5f703f1955 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:59.131155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4700d-74d9-4f76-8c00-ce5f703f1955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:59.131155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4700d-74d9-4f76-8c00-ce5f703f1955 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:59.131940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 150/309] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:40:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:40:59.137148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bccac0-c221-451f-8aab-2e139ff95c0e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:59.137148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bccac0-c221-451f-8aab-2e139ff95c0e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:59.137148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bccac0-c221-451f-8aab-2e139ff95c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:59.153423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-76bccac0-c221-451f-8aab-2e139ff95c0e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:40:59.154674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-76bccac0-c221-451f-8aab-2e139ff95c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:40:59.190137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:59.205699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8b69f-2b1b-4fde-beb2-f470a899ae9b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:59.234826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 151/310] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:40:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:40:59.235760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bccac0-c221-451f-8aab-2e139ff95c0e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:59.259676 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fede6e-992e-4066-9d62-8d6c578565bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:59.260500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fede6e-992e-4066-9d62-8d6c578565bb None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:59.260834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fede6e-992e-4066-9d62-8d6c578565bb None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:59.261729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fede6e-992e-4066-9d62-8d6c578565bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:59.262577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fede6e-992e-4066-9d62-8d6c578565bb None admin] RBAC: Adding query filter params (role.id=86259b0903754e10b0da0ff74bbfa7f8, user.id=50c39f084928467b9e0bcf09edf9941e, scope.system=all) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:59.263170 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fede6e-992e-4066-9d62-8d6c578565bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:59.264330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fede6e-992e-4066-9d62-8d6c578565bb None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:59.270035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 160/311] 10.222.0.17 () {56 vars in 1308 bytes} [Sun Jul 24 19:40:59 2022] GET /identity/v3/role_assignments?user.id=50c39f084928467b9e0bcf09edf9941e&scope.system=all&role.id=86259b0903754e10b0da0ff74bbfa7f8 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:59.281153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fede6e-992e-4066-9d62-8d6c578565bb None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:59.303627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:59.351430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bccac0-c221-451f-8aab-2e139ff95c0e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:59.396677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e6d0c-bb15-4ac8-8e9f-2dd34de1a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:59.397914 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e6d0c-bb15-4ac8-8e9f-2dd34de1a298 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:59.398398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e6d0c-bb15-4ac8-8e9f-2dd34de1a298 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:59.398869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e6d0c-bb15-4ac8-8e9f-2dd34de1a298 None admin] PATH_INFO: `/v3/groups` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:59.399791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e6d0c-bb15-4ac8-8e9f-2dd34de1a298 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:59.401679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e6d0c-bb15-4ac8-8e9f-2dd34de1a298 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:59.414603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fede6e-992e-4066-9d62-8d6c578565bb None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:59.453487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd09dc-de5f-481b-8a81-302bf2bcee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:59.454637 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd09dc-de5f-481b-8a81-302bf2bcee4a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:59.455114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd09dc-de5f-481b-8a81-302bf2bcee4a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:59.455604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd09dc-de5f-481b-8a81-302bf2bcee4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:59.467563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd09dc-de5f-481b-8a81-302bf2bcee4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-c7fd09dc-de5f-481b-8a81-302bf2bcee4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:40:59.475818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:40:59.476976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:40:59.477749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 161/312] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:40:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:40:59.485713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd09dc-de5f-481b-8a81-302bf2bcee4a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:59.490906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 152/313] 10.222.0.17 () {60 vars in 1134 bytes} [Sun Jul 24 19:40:59 2022] POST /identity/v3/groups => generated 229 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:40:59.584291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:59.622453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd09dc-de5f-481b-8a81-302bf2bcee4a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:59.663024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d83522-68fa-443c-b1db-b7fe361b4764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:59.663624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d83522-68fa-443c-b1db-b7fe361b4764 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:59.663907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d83522-68fa-443c-b1db-b7fe361b4764 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:59.664180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d83522-68fa-443c-b1db-b7fe361b4764 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:59.664975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d83522-68fa-443c-b1db-b7fe361b4764 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:40:59.665277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d83522-68fa-443c-b1db-b7fe361b4764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:59.667069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d83522-68fa-443c-b1db-b7fe361b4764 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:59.677244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 162/314] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:40:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:59.684314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e6d0c-bb15-4ac8-8e9f-2dd34de1a298 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:59.718428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:40:59.804307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e6d0c-bb15-4ac8-8e9f-2dd34de1a298 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:40:59.843225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1923727-99b8-4baf-830c-983488f1ee6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:40:59.844135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1923727-99b8-4baf-830c-983488f1ee6f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:40:59.844413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1923727-99b8-4baf-830c-983488f1ee6f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:40:59.844717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1923727-99b8-4baf-830c-983488f1ee6f None admin] PATH_INFO: `/v3/users/735e64813dcd4d5ba7823db0f8fdade9` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:40:59.846056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1923727-99b8-4baf-830c-983488f1ee6f None admin] RBAC: Authorizing `identity:get_user(user_id=735e64813dcd4d5ba7823db0f8fdade9)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:40:59.847531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1923727-99b8-4baf-830c-983488f1ee6f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:40:59.849697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 153/315] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:40:59 2022] GET /identity/v3/users/735e64813dcd4d5ba7823db0f8fdade9 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:40:59.858770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d83522-68fa-443c-b1db-b7fe361b4764 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:40:59.975456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d83522-68fa-443c-b1db-b7fe361b4764 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:00.015544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cddda-a4a9-4c0c-8d1f-a66ef5f09714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:00.016207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cddda-a4a9-4c0c-8d1f-a66ef5f09714 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:00.016496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cddda-a4a9-4c0c-8d1f-a66ef5f09714 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:00.016812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cddda-a4a9-4c0c-8d1f-a66ef5f09714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:00.019666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:00.098374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cddda-a4a9-4c0c-8d1f-a66ef5f09714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:00.099919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cddda-a4a9-4c0c-8d1f-a66ef5f09714 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:00.212433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 163/316] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:00.218404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1923727-99b8-4baf-830c-983488f1ee6f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:00.317652 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:00.325121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1923727-99b8-4baf-830c-983488f1ee6f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:00.362002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680e24c-dc4c-499b-9934-9eeac192aae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:00.362878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680e24c-dc4c-499b-9934-9eeac192aae8 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:00.363161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680e24c-dc4c-499b-9934-9eeac192aae8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:00.363434 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680e24c-dc4c-499b-9934-9eeac192aae8 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:00.364636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680e24c-dc4c-499b-9934-9eeac192aae8 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:00.366116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680e24c-dc4c-499b-9934-9eeac192aae8 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:00.368132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 154/317] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:00 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:00.374029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cddda-a4a9-4c0c-8d1f-a66ef5f09714 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:00.417329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:00.494942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cddda-a4a9-4c0c-8d1f-a66ef5f09714 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:00.532677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a383be-c6f7-4fb3-82bf-a5926f748959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:00.533370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a383be-c6f7-4fb3-82bf-a5926f748959 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:00.533658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a383be-c6f7-4fb3-82bf-a5926f748959 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:00.533935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a383be-c6f7-4fb3-82bf-a5926f748959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:00.534789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a383be-c6f7-4fb3-82bf-a5926f748959 None admin] RBAC: Adding query filter params (role.id=86259b0903754e10b0da0ff74bbfa7f8, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc, user.id=735e64813dcd4d5ba7823db0f8fdade9) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:00.535106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a383be-c6f7-4fb3-82bf-a5926f748959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:00.536566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a383be-c6f7-4fb3-82bf-a5926f748959 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:00.543895 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 164/318] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:00 2022] GET /identity/v3/role_assignments?user.id=735e64813dcd4d5ba7823db0f8fdade9&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=86259b0903754e10b0da0ff74bbfa7f8 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:01.017280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa41e-3ddd-474c-b8a0-00607b2c3b5f None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.017652 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa41e-3ddd-474c-b8a0-00607b2c3b5f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.017851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa41e-3ddd-474c-b8a0-00607b2c3b5f None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.019358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 155/319] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:01.024238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcb405-8fe4-4024-8825-db63223d57b2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.024769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcb405-8fe4-4024-8825-db63223d57b2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.024856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcb405-8fe4-4024-8825-db63223d57b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.041124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-82bcb405-8fe4-4024-8825-db63223d57b2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:01.043094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-82bcb405-8fe4-4024-8825-db63223d57b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:01.111272 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 165/320] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:01.134747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fa41e-3ddd-474c-b8a0-00607b2c3b5f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:01.236736 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fa41e-3ddd-474c-b8a0-00607b2c3b5f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:01.270552 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78288a1b-7bc8-469f-945d-f1b13ef44c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:01.271268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78288a1b-7bc8-469f-945d-f1b13ef44c9e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.271548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78288a1b-7bc8-469f-945d-f1b13ef44c9e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.271822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78288a1b-7bc8-469f-945d-f1b13ef44c9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.282057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78288a1b-7bc8-469f-945d-f1b13ef44c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-78288a1b-7bc8-469f-945d-f1b13ef44c9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:01.289762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:01.290660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:01.291446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:01.293578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 156/321] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:01.297536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcb405-8fe4-4024-8825-db63223d57b2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:01.325921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:01.389634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a560b76-b1de-4036-8f26-5e051cf69df3 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.389944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a560b76-b1de-4036-8f26-5e051cf69df3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.390187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a560b76-b1de-4036-8f26-5e051cf69df3 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.391546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 157/322] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:41:01.398144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d96f1f-828a-4465-ad1b-c911de49f163 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.398428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d96f1f-828a-4465-ad1b-c911de49f163 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.398716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d96f1f-828a-4465-ad1b-c911de49f163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.402812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcb405-8fe4-4024-8825-db63223d57b2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:01.419057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-11d96f1f-828a-4465-ad1b-c911de49f163 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:01.420254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-11d96f1f-828a-4465-ad1b-c911de49f163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:01.440632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:01.441930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fa1e6-e70e-4288-ac95-4a67b7d441f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:01.442611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fa1e6-e70e-4288-ac95-4a67b7d441f7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.442900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fa1e6-e70e-4288-ac95-4a67b7d441f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.443178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fa1e6-e70e-4288-ac95-4a67b7d441f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.443935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fa1e6-e70e-4288-ac95-4a67b7d441f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:01.444245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fa1e6-e70e-4288-ac95-4a67b7d441f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:01.445682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fa1e6-e70e-4288-ac95-4a67b7d441f7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:01.458700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 166/323] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:41:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:01.459750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fa1e6-e70e-4288-ac95-4a67b7d441f7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:01.508361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 158/324] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:01.531253 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d96f1f-828a-4465-ad1b-c911de49f163 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:01.567299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:01.577756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fa1e6-e70e-4288-ac95-4a67b7d441f7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:01.613724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f368fa-29f1-4b9c-abb6-515bd7513383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:01.614602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f368fa-29f1-4b9c-abb6-515bd7513383 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.614884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f368fa-29f1-4b9c-abb6-515bd7513383 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.615165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f368fa-29f1-4b9c-abb6-515bd7513383 None admin] PATH_INFO: `/v3/users/342a844efb1948128076c24c198c9d89` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.616354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f368fa-29f1-4b9c-abb6-515bd7513383 None admin] RBAC: Authorizing `identity:get_user(user_id=342a844efb1948128076c24c198c9d89)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:01.617828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f368fa-29f1-4b9c-abb6-515bd7513383 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:01.624304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 167/325] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:41:01 2022] GET /identity/v3/users/342a844efb1948128076c24c198c9d89 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:01.625376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f368fa-29f1-4b9c-abb6-515bd7513383 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:01.647248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d96f1f-828a-4465-ad1b-c911de49f163 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:01.688752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1942dd-757e-4525-b5cc-ece6b4eb0dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:01.689853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1942dd-757e-4525-b5cc-ece6b4eb0dcd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.690311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1942dd-757e-4525-b5cc-ece6b4eb0dcd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.690756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1942dd-757e-4525-b5cc-ece6b4eb0dcd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.702899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1942dd-757e-4525-b5cc-ece6b4eb0dcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-0c1942dd-757e-4525-b5cc-ece6b4eb0dcd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:01.710586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:01.711473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:01.712450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 159/326] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:01.713329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8e6f9-40c9-4f5e-a882-41cc2fd674f5 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.713692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8e6f9-40c9-4f5e-a882-41cc2fd674f5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.713929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8e6f9-40c9-4f5e-a882-41cc2fd674f5 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.716678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 160/327] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:01.718583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8e6f9-40c9-4f5e-a882-41cc2fd674f5 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:01.735305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:01.736514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f368fa-29f1-4b9c-abb6-515bd7513383 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:01.770617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8706dc3-0b68-45f1-842e-ea428b1f24ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:01.771267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8706dc3-0b68-45f1-842e-ea428b1f24ba None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.771548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8706dc3-0b68-45f1-842e-ea428b1f24ba None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.771822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8706dc3-0b68-45f1-842e-ea428b1f24ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.772679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8706dc3-0b68-45f1-842e-ea428b1f24ba None admin] RBAC: Adding query filter params (role.id=2a20315693ce44d696f86173c0f6eb09, user.id=342a844efb1948128076c24c198c9d89, scope.system=all) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:01.772986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8706dc3-0b68-45f1-842e-ea428b1f24ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:01.774417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8706dc3-0b68-45f1-842e-ea428b1f24ba None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:01.786467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 168/328] 10.222.0.17 () {56 vars in 1308 bytes} [Sun Jul 24 19:41:01 2022] GET /identity/v3/role_assignments?user.id=342a844efb1948128076c24c198c9d89&scope.system=all&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:01.787623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbea5f1-d102-4792-9378-9709e2d14f03 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.787913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbea5f1-d102-4792-9378-9709e2d14f03 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.788188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbea5f1-d102-4792-9378-9709e2d14f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.805907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-0bbea5f1-d102-4792-9378-9709e2d14f03 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:01.807161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0bbea5f1-d102-4792-9378-9709e2d14f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:01.819480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:01.850548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8e6f9-40c9-4f5e-a882-41cc2fd674f5 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:01.904683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 169/329] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:01.906846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fbbaf-3597-493a-aed5-69c0bdae0dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:01.907543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fbbaf-3597-493a-aed5-69c0bdae0dd2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:01.907824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fbbaf-3597-493a-aed5-69c0bdae0dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:01.908099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fbbaf-3597-493a-aed5-69c0bdae0dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:01.908892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fbbaf-3597-493a-aed5-69c0bdae0dd2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:01.909197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fbbaf-3597-493a-aed5-69c0bdae0dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:01.910611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fbbaf-3597-493a-aed5-69c0bdae0dd2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:01.920414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 161/330] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:41:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:01.922307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbea5f1-d102-4792-9378-9709e2d14f03 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:01.926256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fbbaf-3597-493a-aed5-69c0bdae0dd2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:02.033756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:02.052447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbea5f1-d102-4792-9378-9709e2d14f03 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:02.059265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fbbaf-3597-493a-aed5-69c0bdae0dd2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:02.087144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641105c-1c06-4573-adc4-89a350172e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:02.087943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641105c-1c06-4573-adc4-89a350172e5d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:02.088227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641105c-1c06-4573-adc4-89a350172e5d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:02.088502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641105c-1c06-4573-adc4-89a350172e5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:02.090266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641105c-1c06-4573-adc4-89a350172e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:02.091719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641105c-1c06-4573-adc4-89a350172e5d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:02.093586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 170/331] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:02.099016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641105c-1c06-4573-adc4-89a350172e5d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:02.105337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a91b4d-dcd0-4a40-8449-0dc34f80b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:02.106208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a91b4d-dcd0-4a40-8449-0dc34f80b05a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:02.106495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a91b4d-dcd0-4a40-8449-0dc34f80b05a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:02.106772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a91b4d-dcd0-4a40-8449-0dc34f80b05a None admin] PATH_INFO: `/v3/users/50c39f084928467b9e0bcf09edf9941e` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:02.107967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a91b4d-dcd0-4a40-8449-0dc34f80b05a None admin] RBAC: Authorizing `identity:get_user(user_id=50c39f084928467b9e0bcf09edf9941e)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:02.109451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a91b4d-dcd0-4a40-8449-0dc34f80b05a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:02.111083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 162/332] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:41:01 2022] GET /identity/v3/users/50c39f084928467b9e0bcf09edf9941e => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:02.114607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:02.116493 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a91b4d-dcd0-4a40-8449-0dc34f80b05a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:02.200840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641105c-1c06-4573-adc4-89a350172e5d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:02.217878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a91b4d-dcd0-4a40-8449-0dc34f80b05a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:02.233965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce787725-2bfe-41ca-961e-c22699b50ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:02.234633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce787725-2bfe-41ca-961e-c22699b50ccd None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:02.234916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce787725-2bfe-41ca-961e-c22699b50ccd None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:02.235192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce787725-2bfe-41ca-961e-c22699b50ccd None admin] PATH_INFO: `/v3/groups` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:02.235885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce787725-2bfe-41ca-961e-c22699b50ccd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:02.237370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce787725-2bfe-41ca-961e-c22699b50ccd None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:02.251254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bb436-a6b2-4fe6-a970-4a2d02368624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:02.252119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bb436-a6b2-4fe6-a970-4a2d02368624 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:02.255631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bb436-a6b2-4fe6-a970-4a2d02368624 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:02.255631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bb436-a6b2-4fe6-a970-4a2d02368624 None admin] PATH_INFO: `/v3/system/users/50c39f084928467b9e0bcf09edf9941e/roles/86259b0903754e10b0da0ff74bbfa7f8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:02.255631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bb436-a6b2-4fe6-a970-4a2d02368624 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=50c39f084928467b9e0bcf09edf9941e, role_id=86259b0903754e10b0da0ff74bbfa7f8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:02.255631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bb436-a6b2-4fe6-a970-4a2d02368624 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:02.335371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 171/333] 10.222.0.17 () {60 vars in 1134 bytes} [Sun Jul 24 19:41:02 2022] POST /identity/v3/groups => generated 226 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:41:02.358865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 163/334] 10.222.0.17 () {58 vars in 1334 bytes} [Sun Jul 24 19:41:02 2022] PUT /identity/v3/system/users/50c39f084928467b9e0bcf09edf9941e/roles/86259b0903754e10b0da0ff74bbfa7f8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 19:41:04.637159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4bd3-9f00-456d-a67f-8f5a6cd4b484 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:04.637667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4bd3-9f00-456d-a67f-8f5a6cd4b484 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:04.637776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4bd3-9f00-456d-a67f-8f5a6cd4b484 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:04.639430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 172/335] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:04.654712 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821274d-6e65-40b0-83a3-4fd459964b4c None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:04.655638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821274d-6e65-40b0-83a3-4fd459964b4c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:04.656125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821274d-6e65-40b0-83a3-4fd459964b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:04.674024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:04.679931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8821274d-6e65-40b0-83a3-4fd459964b4c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:04.681122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8821274d-6e65-40b0-83a3-4fd459964b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:04.685166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f473755-0e25-41d9-88ec-1269123be697 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:04.685459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f473755-0e25-41d9-88ec-1269123be697 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:04.685740 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f473755-0e25-41d9-88ec-1269123be697 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:04.687119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 173/336] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:04.693651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c5450-974a-4e73-b405-acf1304a0578 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:04.694180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c5450-974a-4e73-b405-acf1304a0578 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:04.694616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c5450-974a-4e73-b405-acf1304a0578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:04.725987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2a0c5450-974a-4e73-b405-acf1304a0578 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:04.727509 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2a0c5450-974a-4e73-b405-acf1304a0578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:04.815114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 164/337] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:04.858181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821274d-6e65-40b0-83a3-4fd459964b4c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:04.860466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 174/338] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:04.884471 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c5450-974a-4e73-b405-acf1304a0578 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:04.888628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:05.002143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821274d-6e65-40b0-83a3-4fd459964b4c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:05.040331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c5450-974a-4e73-b405-acf1304a0578 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:05.079021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499ed2e-08c5-4472-95ee-ef8a2e7511a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:05.079865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499ed2e-08c5-4472-95ee-ef8a2e7511a3 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.080147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499ed2e-08c5-4472-95ee-ef8a2e7511a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.080426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499ed2e-08c5-4472-95ee-ef8a2e7511a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.094180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:05.110108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499ed2e-08c5-4472-95ee-ef8a2e7511a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-a499ed2e-08c5-4472-95ee-ef8a2e7511a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:05.118922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:05.119844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 24 19:41:05.121211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:05.123553 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 165/339] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:05.127522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658d706-81b1-44db-ada8-d33e95e4f06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:05.128934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658d706-81b1-44db-ada8-d33e95e4f06e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.129223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658d706-81b1-44db-ada8-d33e95e4f06e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.129427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499ed2e-08c5-4472-95ee-ef8a2e7511a3 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:05.129527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658d706-81b1-44db-ada8-d33e95e4f06e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.160703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658d706-81b1-44db-ada8-d33e95e4f06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-9658d706-81b1-44db-ada8-d33e95e4f06e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:05.169035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:05.171450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 175/340] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:05.173414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f5041-c8ec-4ed2-900f-f2d96812a8ab None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.175101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f5041-c8ec-4ed2-900f-f2d96812a8ab None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.175101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f5041-c8ec-4ed2-900f-f2d96812a8ab None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.175101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 176/341] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:05.181641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3205a-0d50-499b-ac1d-576d6191ceb9 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.182276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3205a-0d50-499b-ac1d-576d6191ceb9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.183314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3205a-0d50-499b-ac1d-576d6191ceb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.223636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-68a3205a-0d50-499b-ac1d-576d6191ceb9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:05.225231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-68a3205a-0d50-499b-ac1d-576d6191ceb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:05.268079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499ed2e-08c5-4472-95ee-ef8a2e7511a3 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:05.310996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fbf07-41a5-4cde-8787-be2c619afbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:05.311755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fbf07-41a5-4cde-8787-be2c619afbf7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.312041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fbf07-41a5-4cde-8787-be2c619afbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.312314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fbf07-41a5-4cde-8787-be2c619afbf7 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.313885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fbf07-41a5-4cde-8787-be2c619afbf7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:05.314198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fbf07-41a5-4cde-8787-be2c619afbf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:05.315941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fbf07-41a5-4cde-8787-be2c619afbf7 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:05.335393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 166/342] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:41:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:05.336259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fbf07-41a5-4cde-8787-be2c619afbf7 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:05.377006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:05.385610 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 177/343] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:05.390404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3205a-0d50-499b-ac1d-576d6191ceb9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:05.463367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:05.518034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fbf07-41a5-4cde-8787-be2c619afbf7 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:05.569461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13764e4-733f-4b12-98f2-dac572254716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:05.570067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13764e4-733f-4b12-98f2-dac572254716 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.570508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13764e4-733f-4b12-98f2-dac572254716 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.570782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13764e4-733f-4b12-98f2-dac572254716 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.572354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13764e4-733f-4b12-98f2-dac572254716 None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:05.574115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13764e4-733f-4b12-98f2-dac572254716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:05.575534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13764e4-733f-4b12-98f2-dac572254716 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:05.589307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 167/344] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:41:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:05.595729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13764e4-733f-4b12-98f2-dac572254716 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:05.610198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3205a-0d50-499b-ac1d-576d6191ceb9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:05.672335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917882e-3dc2-47a0-9031-0c9946e01a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:05.673256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917882e-3dc2-47a0-9031-0c9946e01a20 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.673636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917882e-3dc2-47a0-9031-0c9946e01a20 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.673922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917882e-3dc2-47a0-9031-0c9946e01a20 None admin] PATH_INFO: `/v3/users/50c39f084928467b9e0bcf09edf9941e` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.675221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917882e-3dc2-47a0-9031-0c9946e01a20 None admin] RBAC: Authorizing `identity:get_user(user_id=50c39f084928467b9e0bcf09edf9941e)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:05.676699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917882e-3dc2-47a0-9031-0c9946e01a20 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:05.678806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 178/345] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:41:05 2022] GET /identity/v3/users/50c39f084928467b9e0bcf09edf9941e => generated 293 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:05.679623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917882e-3dc2-47a0-9031-0c9946e01a20 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:05.738208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13764e4-733f-4b12-98f2-dac572254716 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:05.749244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:05.764172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:05.782959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a2bc2-5983-4636-9db8-fbf8d60e64d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:05.783797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a2bc2-5983-4636-9db8-fbf8d60e64d8 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.784216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a2bc2-5983-4636-9db8-fbf8d60e64d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.784511 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a2bc2-5983-4636-9db8-fbf8d60e64d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.797019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917882e-3dc2-47a0-9031-0c9946e01a20 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:05.800573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a2bc2-5983-4636-9db8-fbf8d60e64d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-735a2bc2-5983-4636-9db8-fbf8d60e64d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:05.810907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:05.811936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:05.812771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 168/346] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:41:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:05.813659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f68138-4005-4bcf-a191-0391be804ba0 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.814091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f68138-4005-4bcf-a191-0391be804ba0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.814795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f68138-4005-4bcf-a191-0391be804ba0 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.816775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 169/347] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:05.817711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f68138-4005-4bcf-a191-0391be804ba0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:05.849635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5a989-8754-4155-be10-3f92aa260b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:05.850557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5a989-8754-4155-be10-3f92aa260b70 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.850941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5a989-8754-4155-be10-3f92aa260b70 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.851552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5a989-8754-4155-be10-3f92aa260b70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.930021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5a989-8754-4155-be10-3f92aa260b70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:05.931656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5a989-8754-4155-be10-3f92aa260b70 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:05.932508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f68138-4005-4bcf-a191-0391be804ba0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:05.972868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402e348-d705-4c8f-97f0-566f1ba3ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:05.973771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402e348-d705-4c8f-97f0-566f1ba3ad4b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:05.974218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402e348-d705-4c8f-97f0-566f1ba3ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:05.974629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402e348-d705-4c8f-97f0-566f1ba3ad4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:05.975596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402e348-d705-4c8f-97f0-566f1ba3ad4b None admin] RBAC: Adding query filter params (role.id=86259b0903754e10b0da0ff74bbfa7f8, user.id=50c39f084928467b9e0bcf09edf9941e, scope.system=all) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:05.976044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402e348-d705-4c8f-97f0-566f1ba3ad4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:05.977667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402e348-d705-4c8f-97f0-566f1ba3ad4b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:05.990668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 170/348] 10.222.0.17 () {56 vars in 1308 bytes} [Sun Jul 24 19:41:05 2022] GET /identity/v3/role_assignments?user.id=50c39f084928467b9e0bcf09edf9941e&scope.system=all&role.id=86259b0903754e10b0da0ff74bbfa7f8 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:05.991762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402e348-d705-4c8f-97f0-566f1ba3ad4b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:06.006639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:06.049938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:06.062357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 179/349] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:06.064265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc5107-24b5-451b-b9e7-3e263005125a None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.064767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc5107-24b5-451b-b9e7-3e263005125a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.065148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc5107-24b5-451b-b9e7-3e263005125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.092085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-b9dc5107-24b5-451b-b9e7-3e263005125a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:06.093818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b9dc5107-24b5-451b-b9e7-3e263005125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:06.184341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 180/350] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:06.185374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc5107-24b5-451b-b9e7-3e263005125a None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:06.240786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402e348-d705-4c8f-97f0-566f1ba3ad4b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:06.307116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1a932-7c29-47d4-ae62-e70733301090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:06.308585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1a932-7c29-47d4-ae62-e70733301090 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.309022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1a932-7c29-47d4-ae62-e70733301090 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.311937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1a932-7c29-47d4-ae62-e70733301090 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.311937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1a932-7c29-47d4-ae62-e70733301090 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:06.311937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1a932-7c29-47d4-ae62-e70733301090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:06.312680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1a932-7c29-47d4-ae62-e70733301090 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:06.324487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 171/351] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:06.326018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1a932-7c29-47d4-ae62-e70733301090 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:06.339441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:06.350868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc5107-24b5-451b-b9e7-3e263005125a None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:06.395555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949e318-ef33-47b5-9647-a27cba9d1947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:06.396786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949e318-ef33-47b5-9647-a27cba9d1947 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.397213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949e318-ef33-47b5-9647-a27cba9d1947 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.397691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949e318-ef33-47b5-9647-a27cba9d1947 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.399117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949e318-ef33-47b5-9647-a27cba9d1947 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:06.400799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949e318-ef33-47b5-9647-a27cba9d1947 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:06.402853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 181/352] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:06 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:06.403980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949e318-ef33-47b5-9647-a27cba9d1947 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:06.462111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:06.465134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1a932-7c29-47d4-ae62-e70733301090 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:06.506157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba404c2-5c5d-4366-9d9c-e947eb796d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:06.507026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba404c2-5c5d-4366-9d9c-e947eb796d6e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.507314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba404c2-5c5d-4366-9d9c-e947eb796d6e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.507586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba404c2-5c5d-4366-9d9c-e947eb796d6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.521329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba404c2-5c5d-4366-9d9c-e947eb796d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-9ba404c2-5c5d-4366-9d9c-e947eb796d6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:06.529312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:06.530243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:06.531023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 172/353] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:06.531679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba404c2-5c5d-4366-9d9c-e947eb796d6e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:06.545260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949e318-ef33-47b5-9647-a27cba9d1947 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:06.586680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752cef7-7d9a-4532-86ca-b13e6b46567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:06.588294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752cef7-7d9a-4532-86ca-b13e6b46567b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.588584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752cef7-7d9a-4532-86ca-b13e6b46567b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.588952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752cef7-7d9a-4532-86ca-b13e6b46567b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.623388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:06.658667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba404c2-5c5d-4366-9d9c-e947eb796d6e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:06.681042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752cef7-7d9a-4532-86ca-b13e6b46567b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:06.682620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752cef7-7d9a-4532-86ca-b13e6b46567b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:06.709498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456fa0c-ee80-4e57-a7f2-88cbca39bba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:06.710435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456fa0c-ee80-4e57-a7f2-88cbca39bba7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.710722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456fa0c-ee80-4e57-a7f2-88cbca39bba7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.710997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456fa0c-ee80-4e57-a7f2-88cbca39bba7 None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.720480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456fa0c-ee80-4e57-a7f2-88cbca39bba7 None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:06.721866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456fa0c-ee80-4e57-a7f2-88cbca39bba7 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:06.723911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 173/354] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:06 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:06.725299 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae3668-a35c-4d3e-b661-48c6ea1132df None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.725937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae3668-a35c-4d3e-b661-48c6ea1132df None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.725937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae3668-a35c-4d3e-b661-48c6ea1132df None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.732136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 174/355] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:06.732136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae3668-a35c-4d3e-b661-48c6ea1132df None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:06.766241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:06.830056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 182/356] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:06.830938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752cef7-7d9a-4532-86ca-b13e6b46567b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:06.864854 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae3668-a35c-4d3e-b661-48c6ea1132df None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:06.903262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:06.904634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9e939-b1f7-4377-ac53-398298681392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:06.905347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9e939-b1f7-4377-ac53-398298681392 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.905637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9e939-b1f7-4377-ac53-398298681392 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.905920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9e939-b1f7-4377-ac53-398298681392 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.906678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9e939-b1f7-4377-ac53-398298681392 None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:06.906992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9e939-b1f7-4377-ac53-398298681392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:06.908398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9e939-b1f7-4377-ac53-398298681392 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:06.922989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 175/357] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:41:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:06.924631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1046e5d-71a4-4c58-8192-5de4331e3330 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.925123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1046e5d-71a4-4c58-8192-5de4331e3330 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.925395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1046e5d-71a4-4c58-8192-5de4331e3330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.944975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e1046e5d-71a4-4c58-8192-5de4331e3330 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:06.946429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e1046e5d-71a4-4c58-8192-5de4331e3330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:06.953686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752cef7-7d9a-4532-86ca-b13e6b46567b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:06.993200 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b3c1d-8ebc-4b5e-9825-fc64fbb25faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:06.993974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b3c1d-8ebc-4b5e-9825-fc64fbb25faf None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:06.994323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b3c1d-8ebc-4b5e-9825-fc64fbb25faf None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:06.994607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b3c1d-8ebc-4b5e-9825-fc64fbb25faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:06.996246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b3c1d-8ebc-4b5e-9825-fc64fbb25faf None admin] RBAC: Adding query filter params (group.id=25f360154b9b475590ae79e7b37e0959, role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:06.996575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b3c1d-8ebc-4b5e-9825-fc64fbb25faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:06.998189 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b3c1d-8ebc-4b5e-9825-fc64fbb25faf None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:07.004780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 183/358] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:06 2022] GET /identity/v3/role_assignments?group.id=25f360154b9b475590ae79e7b37e0959&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:07.005385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b3c1d-8ebc-4b5e-9825-fc64fbb25faf None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:07.047945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 176/359] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:07.049218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e3ac-412f-4ea2-b32d-7800c3dc5453 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:07.049513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e3ac-412f-4ea2-b32d-7800c3dc5453 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:07.049809 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e3ac-412f-4ea2-b32d-7800c3dc5453 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:07.051432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 177/360] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:07.052185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95e3ac-412f-4ea2-b32d-7800c3dc5453 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:07.081201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:07.162974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b3c1d-8ebc-4b5e-9825-fc64fbb25faf None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:07.194543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95e3ac-412f-4ea2-b32d-7800c3dc5453 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:07.197360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:07.212670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db36417-949b-408b-a489-02bab317a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:07.213567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db36417-949b-408b-a489-02bab317a702 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:07.213900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db36417-949b-408b-a489-02bab317a702 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:07.214741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db36417-949b-408b-a489-02bab317a702 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:07.216112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db36417-949b-408b-a489-02bab317a702 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:07.217765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db36417-949b-408b-a489-02bab317a702 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:07.219702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 184/361] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:07 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:07.221006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d30cb-2c0d-4ab4-9f02-b99f6322e862 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:07.221292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d30cb-2c0d-4ab4-9f02-b99f6322e862 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:07.221570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d30cb-2c0d-4ab4-9f02-b99f6322e862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:07.241703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-7a9d30cb-2c0d-4ab4-9f02-b99f6322e862 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:07.242866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7a9d30cb-2c0d-4ab4-9f02-b99f6322e862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:07.243510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedd4d6-f439-4940-92ae-ff441239410b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:07.244273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedd4d6-f439-4940-92ae-ff441239410b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:07.244618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedd4d6-f439-4940-92ae-ff441239410b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:07.244934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedd4d6-f439-4940-92ae-ff441239410b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:07.329397 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 185/362] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:07.330370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d30cb-2c0d-4ab4-9f02-b99f6322e862 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:07.334426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedd4d6-f439-4940-92ae-ff441239410b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:07.336131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedd4d6-f439-4940-92ae-ff441239410b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:07.357395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:07.471070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d30cb-2c0d-4ab4-9f02-b99f6322e862 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:07.479829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 178/363] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:07.480729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedd4d6-f439-4940-92ae-ff441239410b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:07.497087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:07.515139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb746170-58dc-4380-ab4d-463dc39d0152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:07.516371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb746170-58dc-4380-ab4d-463dc39d0152 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:07.517001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb746170-58dc-4380-ab4d-463dc39d0152 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:07.517461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb746170-58dc-4380-ab4d-463dc39d0152 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:07.535913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb746170-58dc-4380-ab4d-463dc39d0152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-eb746170-58dc-4380-ab4d-463dc39d0152 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:07.544321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:07.545290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:07.546086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:07.547897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 186/364] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:41:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:07.548542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb746170-58dc-4380-ab4d-463dc39d0152 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:07.613880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedd4d6-f439-4940-92ae-ff441239410b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:07.636454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:07.656570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489870f5-d0aa-4b96-8c53-c6d534780c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:07.657500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489870f5-d0aa-4b96-8c53-c6d534780c2a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:07.657904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489870f5-d0aa-4b96-8c53-c6d534780c2a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:07.658061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489870f5-d0aa-4b96-8c53-c6d534780c2a None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:07.659373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489870f5-d0aa-4b96-8c53-c6d534780c2a None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:07.660978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489870f5-d0aa-4b96-8c53-c6d534780c2a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:07.663705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 179/365] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:07 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:07.663837 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489870f5-d0aa-4b96-8c53-c6d534780c2a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:07.690037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb746170-58dc-4380-ab4d-463dc39d0152 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:07.731545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84feb9-5498-45b7-b05d-090e8cee0f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:07.732352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84feb9-5498-45b7-b05d-090e8cee0f90 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:07.732635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84feb9-5498-45b7-b05d-090e8cee0f90 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:07.732941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84feb9-5498-45b7-b05d-090e8cee0f90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:07.746096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84feb9-5498-45b7-b05d-090e8cee0f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-6f84feb9-5498-45b7-b05d-090e8cee0f90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:07.757138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:07.758263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:07.759065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 187/366] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:07.763960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84feb9-5498-45b7-b05d-090e8cee0f90 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:07.817427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489870f5-d0aa-4b96-8c53-c6d534780c2a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:07.824248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:07.863052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73877383-02cf-4271-9453-7b00b2767a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:07.864173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73877383-02cf-4271-9453-7b00b2767a94 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:07.864606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73877383-02cf-4271-9453-7b00b2767a94 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:07.867518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73877383-02cf-4271-9453-7b00b2767a94 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:07.869161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73877383-02cf-4271-9453-7b00b2767a94 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:07.871389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73877383-02cf-4271-9453-7b00b2767a94 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:07.874458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 180/367] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:07 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:07.874458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73877383-02cf-4271-9453-7b00b2767a94 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:07.915081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84feb9-5498-45b7-b05d-090e8cee0f90 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:07.944419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:07.963123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2c8f2-a6ba-46df-9c69-46460340258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:07.964019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2c8f2-a6ba-46df-9c69-46460340258c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:07.964322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2c8f2-a6ba-46df-9c69-46460340258c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:07.965472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2c8f2-a6ba-46df-9c69-46460340258c None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:07.966277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2c8f2-a6ba-46df-9c69-46460340258c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:07.966597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2c8f2-a6ba-46df-9c69-46460340258c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:07.968003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2c8f2-a6ba-46df-9c69-46460340258c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:07.977943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 188/368] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:07.979263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2c8f2-a6ba-46df-9c69-46460340258c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:08.014230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73877383-02cf-4271-9453-7b00b2767a94 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:08.065594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc27e9e-a551-4a97-b445-3bcfe7bc2402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:08.065827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc27e9e-a551-4a97-b445-3bcfe7bc2402 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:08.066104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc27e9e-a551-4a97-b445-3bcfe7bc2402 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:08.066388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc27e9e-a551-4a97-b445-3bcfe7bc2402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:08.067707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc27e9e-a551-4a97-b445-3bcfe7bc2402 None admin] RBAC: Adding query filter params (group.id=25f360154b9b475590ae79e7b37e0959, role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:08.067707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc27e9e-a551-4a97-b445-3bcfe7bc2402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:08.071778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc27e9e-a551-4a97-b445-3bcfe7bc2402 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:08.075314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 181/369] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:07 2022] GET /identity/v3/role_assignments?group.id=25f360154b9b475590ae79e7b37e0959&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:08.076160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc27e9e-a551-4a97-b445-3bcfe7bc2402 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:08.140885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2c8f2-a6ba-46df-9c69-46460340258c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:08.145020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:08.197604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a19e29-1824-4035-a836-d9244f15b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:08.198370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a19e29-1824-4035-a836-d9244f15b091 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:08.198661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a19e29-1824-4035-a836-d9244f15b091 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:08.198943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a19e29-1824-4035-a836-d9244f15b091 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:08.199742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a19e29-1824-4035-a836-d9244f15b091 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:08.200057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a19e29-1824-4035-a836-d9244f15b091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:08.201579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a19e29-1824-4035-a836-d9244f15b091 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:08.211797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 189/370] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:08.212904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a19e29-1824-4035-a836-d9244f15b091 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:08.226963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc27e9e-a551-4a97-b445-3bcfe7bc2402 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:08.255439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:08.280217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03984329-d3d4-4deb-a87b-edadb0e9e35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:08.281872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03984329-d3d4-4deb-a87b-edadb0e9e35d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:08.282187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03984329-d3d4-4deb-a87b-edadb0e9e35d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:08.282472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03984329-d3d4-4deb-a87b-edadb0e9e35d None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:08.283761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03984329-d3d4-4deb-a87b-edadb0e9e35d None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:08.285336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03984329-d3d4-4deb-a87b-edadb0e9e35d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:08.286893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 182/371] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:08 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:08.287719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03984329-d3d4-4deb-a87b-edadb0e9e35d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:08.366002 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a19e29-1824-4035-a836-d9244f15b091 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:08.405297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f352f-d285-4265-829d-f0de2484b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:08.405499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f352f-d285-4265-829d-f0de2484b634 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:08.405729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f352f-d285-4265-829d-f0de2484b634 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:08.406004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f352f-d285-4265-829d-f0de2484b634 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:08.452843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03984329-d3d4-4deb-a87b-edadb0e9e35d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:08.457581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:08.496495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9a867-20ed-44ff-8be8-e14fb0be406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:08.497271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9a867-20ed-44ff-8be8-e14fb0be406f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:08.497557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9a867-20ed-44ff-8be8-e14fb0be406f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:08.497829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9a867-20ed-44ff-8be8-e14fb0be406f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:08.507708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f352f-d285-4265-829d-f0de2484b634 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:08.509910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f352f-d285-4265-829d-f0de2484b634 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:08.580411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:08.601786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9a867-20ed-44ff-8be8-e14fb0be406f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:08.603667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9a867-20ed-44ff-8be8-e14fb0be406f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:08.653823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 190/372] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:08.654689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f352f-d285-4265-829d-f0de2484b634 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:08.745600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:08.760000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 183/373] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:08.761420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9a867-20ed-44ff-8be8-e14fb0be406f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:08.789845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f352f-d285-4265-829d-f0de2484b634 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:08.832553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd4b99-2d72-4610-9fc6-733fc23be40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:08.833234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd4b99-2d72-4610-9fc6-733fc23be40c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:08.833528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd4b99-2d72-4610-9fc6-733fc23be40c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:08.833810 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd4b99-2d72-4610-9fc6-733fc23be40c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:08.834779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd4b99-2d72-4610-9fc6-733fc23be40c None admin] RBAC: Adding query filter params (group.id=25f360154b9b475590ae79e7b37e0959, role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:08.835201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd4b99-2d72-4610-9fc6-733fc23be40c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:08.836694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd4b99-2d72-4610-9fc6-733fc23be40c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:08.844163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 191/374] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:08 2022] GET /identity/v3/role_assignments?group.id=25f360154b9b475590ae79e7b37e0959&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:08.845080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd4b99-2d72-4610-9fc6-733fc23be40c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:08.889090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9a867-20ed-44ff-8be8-e14fb0be406f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:08.905101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:08.931715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e483a7-0e1f-44bc-8ab0-74563a225462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:08.933188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e483a7-0e1f-44bc-8ab0-74563a225462 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:08.933532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e483a7-0e1f-44bc-8ab0-74563a225462 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:08.933810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e483a7-0e1f-44bc-8ab0-74563a225462 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:08.935778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e483a7-0e1f-44bc-8ab0-74563a225462 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:08.937388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e483a7-0e1f-44bc-8ab0-74563a225462 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:08.939501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 184/375] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:08 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:08.945224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e483a7-0e1f-44bc-8ab0-74563a225462 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:09.067144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e483a7-0e1f-44bc-8ab0-74563a225462 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:09.112424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cc8cd-07ef-44cf-bdca-3e82543bbff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:09.113534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cc8cd-07ef-44cf-bdca-3e82543bbff6 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:09.113807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cc8cd-07ef-44cf-bdca-3e82543bbff6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:09.114087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cc8cd-07ef-44cf-bdca-3e82543bbff6 None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:09.115332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cc8cd-07ef-44cf-bdca-3e82543bbff6 None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:09.116811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cc8cd-07ef-44cf-bdca-3e82543bbff6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:09.119431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 185/376] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:08 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:09.129701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cc8cd-07ef-44cf-bdca-3e82543bbff6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:09.148587 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd4b99-2d72-4610-9fc6-733fc23be40c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:09.173456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:09.194923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:09.203739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1969741-8800-40bf-b5b9-a4d4335c7808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:09.204768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1969741-8800-40bf-b5b9-a4d4335c7808 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:09.205252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1969741-8800-40bf-b5b9-a4d4335c7808 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:09.205642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1969741-8800-40bf-b5b9-a4d4335c7808 None admin] PATH_INFO: `/v3/projects/245371efe76c48ce9eb35e790603face` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:09.207134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1969741-8800-40bf-b5b9-a4d4335c7808 None admin] RBAC: Authorizing `identity:get_project(project_id=245371efe76c48ce9eb35e790603face)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:09.212020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1969741-8800-40bf-b5b9-a4d4335c7808 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:09.212020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 192/377] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:08 2022] GET /identity/v3/projects/245371efe76c48ce9eb35e790603face => generated 344 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:09.221704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1969741-8800-40bf-b5b9-a4d4335c7808 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:09.275893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cc8cd-07ef-44cf-bdca-3e82543bbff6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:09.328678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bef30c-44f5-4c40-802a-5b613e46f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:09.328678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bef30c-44f5-4c40-802a-5b613e46f529 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:09.328678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bef30c-44f5-4c40-802a-5b613e46f529 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:09.328678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bef30c-44f5-4c40-802a-5b613e46f529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:09.329126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bef30c-44f5-4c40-802a-5b613e46f529 None admin] RBAC: Adding query filter params (group.id=25f360154b9b475590ae79e7b37e0959, role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:09.330553 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bef30c-44f5-4c40-802a-5b613e46f529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:09.331681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bef30c-44f5-4c40-802a-5b613e46f529 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:09.338936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 186/378] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:09 2022] GET /identity/v3/role_assignments?group.id=25f360154b9b475590ae79e7b37e0959&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:09.340294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93131f65-f236-4684-ba7b-04c50fd9f761 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:09.340590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93131f65-f236-4684-ba7b-04c50fd9f761 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:09.340919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93131f65-f236-4684-ba7b-04c50fd9f761 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:09.342502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 187/379] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:09.348340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15bb8c-c8c3-4107-b709-1c20e398ad72 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:09.348676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15bb8c-c8c3-4107-b709-1c20e398ad72 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:09.348945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15bb8c-c8c3-4107-b709-1c20e398ad72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:09.358041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1969741-8800-40bf-b5b9-a4d4335c7808 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:09.368769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-5a15bb8c-c8c3-4107-b709-1c20e398ad72 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:09.370004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5a15bb8c-c8c3-4107-b709-1c20e398ad72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:09.402491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21a3a2-c580-400e-8da0-0a8cf93851e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:09.403549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21a3a2-c580-400e-8da0-0a8cf93851e5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:09.403838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21a3a2-c580-400e-8da0-0a8cf93851e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:09.404124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21a3a2-c580-400e-8da0-0a8cf93851e5 None admin] PATH_INFO: `/v3/groups/7360427643e84f4b8de24daaf707fb19` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:09.414125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21a3a2-c580-400e-8da0-0a8cf93851e5 None admin] RBAC: Authorizing `identity:get_group(group_id=7360427643e84f4b8de24daaf707fb19)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:09.415907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21a3a2-c580-400e-8da0-0a8cf93851e5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:09.421179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 193/380] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:09 2022] GET /identity/v3/groups/7360427643e84f4b8de24daaf707fb19 => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:09.429310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21a3a2-c580-400e-8da0-0a8cf93851e5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:09.476558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 188/381] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:09.506570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15bb8c-c8c3-4107-b709-1c20e398ad72 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:09.521614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:09.527495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:09.606947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21a3a2-c580-400e-8da0-0a8cf93851e5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:09.650777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107a3b0-75eb-42fe-a90b-d73ddccc4a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:09.651456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107a3b0-75eb-42fe-a90b-d73ddccc4a19 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:09.651762 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107a3b0-75eb-42fe-a90b-d73ddccc4a19 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:09.652052 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107a3b0-75eb-42fe-a90b-d73ddccc4a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:09.652933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107a3b0-75eb-42fe-a90b-d73ddccc4a19 None admin] RBAC: Adding query filter params (group.id=7360427643e84f4b8de24daaf707fb19, role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=245371efe76c48ce9eb35e790603face) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:09.653276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107a3b0-75eb-42fe-a90b-d73ddccc4a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:09.654728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107a3b0-75eb-42fe-a90b-d73ddccc4a19 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:09.661342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 194/382] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:09 2022] GET /identity/v3/role_assignments?group.id=7360427643e84f4b8de24daaf707fb19&scope.project.id=245371efe76c48ce9eb35e790603face&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:09.716143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15bb8c-c8c3-4107-b709-1c20e398ad72 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:09.766415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5def324-614a-4c52-a0f7-5a3daf5c8f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:09.767500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5def324-614a-4c52-a0f7-5a3daf5c8f04 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:09.767929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5def324-614a-4c52-a0f7-5a3daf5c8f04 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:09.768358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5def324-614a-4c52-a0f7-5a3daf5c8f04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:09.787104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5def324-614a-4c52-a0f7-5a3daf5c8f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-b5def324-614a-4c52-a0f7-5a3daf5c8f04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:09.795680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:09.796863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:09.797880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:09.800303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 189/383] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:09.806303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107a3b0-75eb-42fe-a90b-d73ddccc4a19 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:09.934834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107a3b0-75eb-42fe-a90b-d73ddccc4a19 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:09.964710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:09.974041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff40f9-6575-4038-b754-12b66421851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:09.986789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff40f9-6575-4038-b754-12b66421851c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:09.986789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff40f9-6575-4038-b754-12b66421851c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:09.986789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff40f9-6575-4038-b754-12b66421851c None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:09.986789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff40f9-6575-4038-b754-12b66421851c None admin] RBAC: Adding query filter params (name=member) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:09.986789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff40f9-6575-4038-b754-12b66421851c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:09.986789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff40f9-6575-4038-b754-12b66421851c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:09.989619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 195/384] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:41:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:09.997747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5def324-614a-4c52-a0f7-5a3daf5c8f04 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:10.117703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:10.149452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5def324-614a-4c52-a0f7-5a3daf5c8f04 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:10.196675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830a895-a8b5-4d41-9939-5503a3014acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:10.197522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830a895-a8b5-4d41-9939-5503a3014acb None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:10.197811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830a895-a8b5-4d41-9939-5503a3014acb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:10.198619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830a895-a8b5-4d41-9939-5503a3014acb None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:10.205416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830a895-a8b5-4d41-9939-5503a3014acb None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:10.206917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830a895-a8b5-4d41-9939-5503a3014acb None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:10.208668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 190/385] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:09 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:10.214808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff40f9-6575-4038-b754-12b66421851c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:10.360906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff40f9-6575-4038-b754-12b66421851c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:10.409569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed80d4e-4752-44b7-b41e-ceae58960864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:10.410478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed80d4e-4752-44b7-b41e-ceae58960864 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:10.410765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed80d4e-4752-44b7-b41e-ceae58960864 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:10.411041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed80d4e-4752-44b7-b41e-ceae58960864 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:10.412396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed80d4e-4752-44b7-b41e-ceae58960864 None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:10.413874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed80d4e-4752-44b7-b41e-ceae58960864 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:10.415945 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 196/386] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:10 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:10.422571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830a895-a8b5-4d41-9939-5503a3014acb None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:10.557237 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830a895-a8b5-4d41-9939-5503a3014acb None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:10.591077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d5059-0764-46b3-bc6d-4e4795ddcfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:10.591993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d5059-0764-46b3-bc6d-4e4795ddcfb2 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:10.592278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d5059-0764-46b3-bc6d-4e4795ddcfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:10.592552 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d5059-0764-46b3-bc6d-4e4795ddcfb2 None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/groups/25f360154b9b475590ae79e7b37e0959/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:10.594602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d5059-0764-46b3-bc6d-4e4795ddcfb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=04a4d4b6451449af851d7f25e1f9c6be, group_id=25f360154b9b475590ae79e7b37e0959, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:10.596088 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d5059-0764-46b3-bc6d-4e4795ddcfb2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:10.695005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 191/387] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Jul 24 19:41:10 2022] PUT /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/groups/25f360154b9b475590ae79e7b37e0959/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:10.758961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bf821-919d-414a-aee3-f51f868a1330 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:10.759295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bf821-919d-414a-aee3-f51f868a1330 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:10.759532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bf821-919d-414a-aee3-f51f868a1330 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:10.761156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 197/388] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:10.766722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3684-5de1-4cb4-b1ca-de62d6faebe2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:10.767013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3684-5de1-4cb4-b1ca-de62d6faebe2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:10.767290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3684-5de1-4cb4-b1ca-de62d6faebe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:10.772581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:10.790257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-4aeb3684-5de1-4cb4-b1ca-de62d6faebe2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:10.791486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4aeb3684-5de1-4cb4-b1ca-de62d6faebe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:10.876333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 192/389] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:10.898457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bf821-919d-414a-aee3-f51f868a1330 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:10.962959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:11.015881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bf821-919d-414a-aee3-f51f868a1330 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:11.065479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3235e60-b284-4543-bcaf-a0366e8109c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:11.066314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3235e60-b284-4543-bcaf-a0366e8109c2 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:11.066596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3235e60-b284-4543-bcaf-a0366e8109c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:11.066870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3235e60-b284-4543-bcaf-a0366e8109c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:11.088974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3235e60-b284-4543-bcaf-a0366e8109c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-c3235e60-b284-4543-bcaf-a0366e8109c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:11.097419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:11.099388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:11.105310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 198/390] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:41:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:11.108384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3684-5de1-4cb4-b1ca-de62d6faebe2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:11.222907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3684-5de1-4cb4-b1ca-de62d6faebe2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:11.254579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:11.259565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08a7ca-26f5-474c-bde0-7ecc05dba80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:11.260246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08a7ca-26f5-474c-bde0-7ecc05dba80a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:11.260551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08a7ca-26f5-474c-bde0-7ecc05dba80a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:11.260855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08a7ca-26f5-474c-bde0-7ecc05dba80a None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:11.261606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08a7ca-26f5-474c-bde0-7ecc05dba80a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:11.261913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08a7ca-26f5-474c-bde0-7ecc05dba80a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:11.263281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08a7ca-26f5-474c-bde0-7ecc05dba80a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:11.272052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 193/391] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:11.278152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3235e60-b284-4543-bcaf-a0366e8109c2 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:11.411251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3235e60-b284-4543-bcaf-a0366e8109c2 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:11.434653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:11.451988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f53a0-8ed6-4035-be25-fdca8abe4d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:11.453287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f53a0-8ed6-4035-be25-fdca8abe4d1d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:11.453767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f53a0-8ed6-4035-be25-fdca8abe4d1d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:11.454265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f53a0-8ed6-4035-be25-fdca8abe4d1d None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:11.455711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f53a0-8ed6-4035-be25-fdca8abe4d1d None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:11.457435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f53a0-8ed6-4035-be25-fdca8abe4d1d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:11.459688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 199/392] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:11 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:11.466215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08a7ca-26f5-474c-bde0-7ecc05dba80a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:11.571797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08a7ca-26f5-474c-bde0-7ecc05dba80a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:11.607875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec0be5-8b0c-45f1-8285-c1564a0f325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:11.609177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec0be5-8b0c-45f1-8285-c1564a0f325d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:11.609660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec0be5-8b0c-45f1-8285-c1564a0f325d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:11.610167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec0be5-8b0c-45f1-8285-c1564a0f325d None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:11.611585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec0be5-8b0c-45f1-8285-c1564a0f325d None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:11.613245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec0be5-8b0c-45f1-8285-c1564a0f325d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:11.615458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 194/393] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:11 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:11.621260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f53a0-8ed6-4035-be25-fdca8abe4d1d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:11.646316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f4074-3f1c-4156-82cc-13dd5e870685 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:11.646957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f4074-3f1c-4156-82cc-13dd5e870685 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:11.647429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f4074-3f1c-4156-82cc-13dd5e870685 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:11.649025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 195/394] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:41:11.656225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257ed8b-dfb1-4117-93d2-f68a837d8bad None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:11.656851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257ed8b-dfb1-4117-93d2-f68a837d8bad None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:11.657333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257ed8b-dfb1-4117-93d2-f68a837d8bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:11.675064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a257ed8b-dfb1-4117-93d2-f68a837d8bad None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:11.676706 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a257ed8b-dfb1-4117-93d2-f68a837d8bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:11.741973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:11.754848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 196/395] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:11.782316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257ed8b-dfb1-4117-93d2-f68a837d8bad None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:11.827463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f53a0-8ed6-4035-be25-fdca8abe4d1d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:11.892343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768e0f2-4c4f-456c-b226-81ad22216abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:11.893645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768e0f2-4c4f-456c-b226-81ad22216abb None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:11.894140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768e0f2-4c4f-456c-b226-81ad22216abb None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:11.894809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768e0f2-4c4f-456c-b226-81ad22216abb None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/groups/25f360154b9b475590ae79e7b37e0959/roles/31a26f4cc3674ca9bdbfc0be860a4318` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:11.897487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768e0f2-4c4f-456c-b226-81ad22216abb None admin] RBAC: Authorizing `identity:create_grant(project_id=04a4d4b6451449af851d7f25e1f9c6be, group_id=25f360154b9b475590ae79e7b37e0959, role_id=31a26f4cc3674ca9bdbfc0be860a4318)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:11.899180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768e0f2-4c4f-456c-b226-81ad22216abb None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:11.937674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257ed8b-dfb1-4117-93d2-f68a837d8bad None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:11.972752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62d836-75e6-45b6-a6b7-546afbd6296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:11.973558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62d836-75e6-45b6-a6b7-546afbd6296c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:11.973892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62d836-75e6-45b6-a6b7-546afbd6296c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:11.974165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62d836-75e6-45b6-a6b7-546afbd6296c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:11.985599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62d836-75e6-45b6-a6b7-546afbd6296c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-5e62d836-75e6-45b6-a6b7-546afbd6296c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:11.993602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:11.994520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:11.995348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 197/396] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:12.002621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62d836-75e6-45b6-a6b7-546afbd6296c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:12.010343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 200/397] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Jul 24 19:41:11 2022] PUT /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be/groups/25f360154b9b475590ae79e7b37e0959/roles/31a26f4cc3674ca9bdbfc0be860a4318 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:12.073451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:12.129260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62d836-75e6-45b6-a6b7-546afbd6296c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:12.169010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921c485-7c9c-439a-94cb-d404311de49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:12.169703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921c485-7c9c-439a-94cb-d404311de49b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.169987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921c485-7c9c-439a-94cb-d404311de49b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.170263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921c485-7c9c-439a-94cb-d404311de49b None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.171031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921c485-7c9c-439a-94cb-d404311de49b None admin] RBAC: Adding query filter params (name=member) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:12.171336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921c485-7c9c-439a-94cb-d404311de49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:12.172795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921c485-7c9c-439a-94cb-d404311de49b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:12.182325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 198/398] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:41:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:12.192598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768e0f2-4c4f-456c-b226-81ad22216abb None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:12.242646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aad71-e512-4a5d-9acc-dd3c96ca9980 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.243317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aad71-e512-4a5d-9acc-dd3c96ca9980 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.243802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aad71-e512-4a5d-9acc-dd3c96ca9980 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.245748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 199/399] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:12.252629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8aa30-114c-4987-8f91-509c61af60e0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.252964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8aa30-114c-4987-8f91-509c61af60e0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.253221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8aa30-114c-4987-8f91-509c61af60e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.260512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:12.274390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6df8aa30-114c-4987-8f91-509c61af60e0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:12.275658 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6df8aa30-114c-4987-8f91-509c61af60e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:12.323783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768e0f2-4c4f-456c-b226-81ad22216abb None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:12.375794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd6506-fb29-4b5f-809b-483e2cf38a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:12.376773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd6506-fb29-4b5f-809b-483e2cf38a9d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.377067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd6506-fb29-4b5f-809b-483e2cf38a9d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.377631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd6506-fb29-4b5f-809b-483e2cf38a9d None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.379124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd6506-fb29-4b5f-809b-483e2cf38a9d None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:12.380695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd6506-fb29-4b5f-809b-483e2cf38a9d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:12.382765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 201/400] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:12 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:12.389784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd6506-fb29-4b5f-809b-483e2cf38a9d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:12.421586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 200/401] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:12.428364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933db50-923f-4461-aac9-61d59502879d None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.428930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933db50-923f-4461-aac9-61d59502879d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.429434 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933db50-923f-4461-aac9-61d59502879d None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.431431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 201/402] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:12.437188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7610ed-09da-4af0-af62-16386c9034d2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.437732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7610ed-09da-4af0-af62-16386c9034d2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.438408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7610ed-09da-4af0-af62-16386c9034d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.458852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8a7610ed-09da-4af0-af62-16386c9034d2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:12.460618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8a7610ed-09da-4af0-af62-16386c9034d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:12.500992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd6506-fb29-4b5f-809b-483e2cf38a9d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:12.507340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:12.515046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:12.536691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59347363-8115-47a9-bc4d-dbb42619710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:12.537635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59347363-8115-47a9-bc4d-dbb42619710a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.537854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59347363-8115-47a9-bc4d-dbb42619710a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.542589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59347363-8115-47a9-bc4d-dbb42619710a None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.542589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59347363-8115-47a9-bc4d-dbb42619710a None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:12.542589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59347363-8115-47a9-bc4d-dbb42619710a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:12.543224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 202/403] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:12.543649 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 202/404] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:12 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:12.544603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7610ed-09da-4af0-af62-16386c9034d2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:12.549389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59347363-8115-47a9-bc4d-dbb42619710a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:12.667088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59347363-8115-47a9-bc4d-dbb42619710a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:12.675223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7610ed-09da-4af0-af62-16386c9034d2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:12.703107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828b8c1-8522-4cd5-abd0-39c8369e0e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:12.704022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828b8c1-8522-4cd5-abd0-39c8369e0e9a None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.704313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828b8c1-8522-4cd5-abd0-39c8369e0e9a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.704598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828b8c1-8522-4cd5-abd0-39c8369e0e9a None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/groups/25f360154b9b475590ae79e7b37e0959/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.706670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828b8c1-8522-4cd5-abd0-39c8369e0e9a None admin] RBAC: Authorizing `identity:create_grant(project_id=6f531532fbb141bd81ca33ecf527a7cc, group_id=25f360154b9b475590ae79e7b37e0959, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:12.708130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828b8c1-8522-4cd5-abd0-39c8369e0e9a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:12.715064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d1bfd-6ab3-4d24-95e2-1784c70ac7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:12.716038 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d1bfd-6ab3-4d24-95e2-1784c70ac7ac None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.716384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d1bfd-6ab3-4d24-95e2-1784c70ac7ac None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.716758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d1bfd-6ab3-4d24-95e2-1784c70ac7ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.727988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d1bfd-6ab3-4d24-95e2-1784c70ac7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-df7d1bfd-6ab3-4d24-95e2-1784c70ac7ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:12.736369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:12.737331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:12.738131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 203/405] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:41:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:12.739092 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d1bfd-6ab3-4d24-95e2-1784c70ac7ac None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:12.779516 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:12.793912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 203/406] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Jul 24 19:41:12 2022] PUT /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/groups/25f360154b9b475590ae79e7b37e0959/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 19:41:12.795700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828b8c1-8522-4cd5-abd0-39c8369e0e9a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:12.858159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:12.937706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828b8c1-8522-4cd5-abd0-39c8369e0e9a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:12.968230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d1bfd-6ab3-4d24-95e2-1784c70ac7ac None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:12.984712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2c45-c9d4-47a7-82a0-e16e96a8fec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:12.986367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2c45-c9d4-47a7-82a0-e16e96a8fec6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:12.987135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2c45-c9d4-47a7-82a0-e16e96a8fec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:12.987608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2c45-c9d4-47a7-82a0-e16e96a8fec6 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:12.988853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a2c45-c9d4-47a7-82a0-e16e96a8fec6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:12.989732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a2c45-c9d4-47a7-82a0-e16e96a8fec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:12.991327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a2c45-c9d4-47a7-82a0-e16e96a8fec6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:13.008795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 204/407] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:13.010949 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2c45-c9d4-47a7-82a0-e16e96a8fec6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:13.017358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3127f-a416-41c0-8107-2b17c96918f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:13.017978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3127f-a416-41c0-8107-2b17c96918f6 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.018264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3127f-a416-41c0-8107-2b17c96918f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.018535 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3127f-a416-41c0-8107-2b17c96918f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.035598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3127f-a416-41c0-8107-2b17c96918f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-4ab3127f-a416-41c0-8107-2b17c96918f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:13.045673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:13.046596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:13.047399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:13.050898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 204/408] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:13.056359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3127f-a416-41c0-8107-2b17c96918f6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:13.158504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2c45-c9d4-47a7-82a0-e16e96a8fec6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:13.160453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:13.202354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc1cdd-e125-42fd-887f-e22bdfc9c856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:13.203920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc1cdd-e125-42fd-887f-e22bdfc9c856 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.207122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc1cdd-e125-42fd-887f-e22bdfc9c856 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.207586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc1cdd-e125-42fd-887f-e22bdfc9c856 None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.209128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc1cdd-e125-42fd-887f-e22bdfc9c856 None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:13.210744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc1cdd-e125-42fd-887f-e22bdfc9c856 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:13.221755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 205/409] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:13 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:13.224998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:13.232605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3127f-a416-41c0-8107-2b17c96918f6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:13.238640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc1cdd-e125-42fd-887f-e22bdfc9c856 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:13.285657 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c7740-11de-422f-b121-9faf5e2d01ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:13.286592 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c7740-11de-422f-b121-9faf5e2d01ab None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.287024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c7740-11de-422f-b121-9faf5e2d01ab None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.287473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c7740-11de-422f-b121-9faf5e2d01ab None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.288452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c7740-11de-422f-b121-9faf5e2d01ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:13.288949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c7740-11de-422f-b121-9faf5e2d01ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:13.290610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c7740-11de-422f-b121-9faf5e2d01ab None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:13.301220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 205/410] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:13.302821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4944b03-040d-40f2-97b2-a50e422552d5 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.303367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4944b03-040d-40f2-97b2-a50e422552d5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.303800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4944b03-040d-40f2-97b2-a50e422552d5 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.305607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 206/411] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:13.310667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4944b03-040d-40f2-97b2-a50e422552d5 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:13.366974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc1cdd-e125-42fd-887f-e22bdfc9c856 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:13.413915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a966cf-aecc-459b-9885-7648371b3398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:13.414820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a966cf-aecc-459b-9885-7648371b3398 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.415107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a966cf-aecc-459b-9885-7648371b3398 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.415390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a966cf-aecc-459b-9885-7648371b3398 None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.417198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a966cf-aecc-459b-9885-7648371b3398 None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:13.419485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a966cf-aecc-459b-9885-7648371b3398 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:13.421956 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 206/412] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:13 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:13.424937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe81c44-593e-49b3-b874-65299fe55e35 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.424937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe81c44-593e-49b3-b874-65299fe55e35 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.424937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe81c44-593e-49b3-b874-65299fe55e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.443475 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-efe81c44-593e-49b3-b874-65299fe55e35 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:13.444942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-efe81c44-593e-49b3-b874-65299fe55e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:13.452757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4944b03-040d-40f2-97b2-a50e422552d5 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:13.482636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:13.502060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32600a-12cd-45aa-942f-ebc3dafd8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:13.503408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32600a-12cd-45aa-942f-ebc3dafd8e46 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.504278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32600a-12cd-45aa-942f-ebc3dafd8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.504731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32600a-12cd-45aa-942f-ebc3dafd8e46 None admin] PATH_INFO: `/v3/groups/7360427643e84f4b8de24daaf707fb19` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.506703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32600a-12cd-45aa-942f-ebc3dafd8e46 None admin] RBAC: Authorizing `identity:get_group(group_id=7360427643e84f4b8de24daaf707fb19)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:13.508857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32600a-12cd-45aa-942f-ebc3dafd8e46 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:13.511962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 207/413] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:13 2022] GET /identity/v3/groups/7360427643e84f4b8de24daaf707fb19 => generated 229 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:13.513013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32600a-12cd-45aa-942f-ebc3dafd8e46 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:13.536675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 207/414] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:13.536675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe81c44-593e-49b3-b874-65299fe55e35 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:13.564819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:13.657901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32600a-12cd-45aa-942f-ebc3dafd8e46 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:13.668727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe81c44-593e-49b3-b874-65299fe55e35 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:13.706757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e7ffc-053e-494d-a19a-aaacf6394b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:13.708462 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e7ffc-053e-494d-a19a-aaacf6394b4e None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.709220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e7ffc-053e-494d-a19a-aaacf6394b4e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.709745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e7ffc-053e-494d-a19a-aaacf6394b4e None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/groups/25f360154b9b475590ae79e7b37e0959/roles/31a26f4cc3674ca9bdbfc0be860a4318` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.712452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e7ffc-053e-494d-a19a-aaacf6394b4e None admin] RBAC: Authorizing `identity:create_grant(project_id=6f531532fbb141bd81ca33ecf527a7cc, group_id=25f360154b9b475590ae79e7b37e0959, role_id=31a26f4cc3674ca9bdbfc0be860a4318)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:13.714731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e7ffc-053e-494d-a19a-aaacf6394b4e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:13.715963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74bbd0-ec23-4253-a0f1-f16d12d0cf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:13.716997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74bbd0-ec23-4253-a0f1-f16d12d0cf9b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.717386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74bbd0-ec23-4253-a0f1-f16d12d0cf9b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.717792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74bbd0-ec23-4253-a0f1-f16d12d0cf9b None admin] PATH_INFO: `/v3/projects/245371efe76c48ce9eb35e790603face` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.721382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74bbd0-ec23-4253-a0f1-f16d12d0cf9b None admin] RBAC: Authorizing `identity:get_project(project_id=245371efe76c48ce9eb35e790603face)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:13.723040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74bbd0-ec23-4253-a0f1-f16d12d0cf9b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:13.725129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 208/415] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:13 2022] GET /identity/v3/projects/245371efe76c48ce9eb35e790603face => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:13.725968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74bbd0-ec23-4253-a0f1-f16d12d0cf9b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:13.765836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:13.812840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 208/416] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Jul 24 19:41:13 2022] PUT /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc/groups/25f360154b9b475590ae79e7b37e0959/roles/31a26f4cc3674ca9bdbfc0be860a4318 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:13.814316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e7ffc-053e-494d-a19a-aaacf6394b4e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:13.882650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74bbd0-ec23-4253-a0f1-f16d12d0cf9b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:13.969571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1144d5-c085-4560-8cd1-c7e941f96af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:13.970312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1144d5-c085-4560-8cd1-c7e941f96af5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:13.970610 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1144d5-c085-4560-8cd1-c7e941f96af5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:13.970891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1144d5-c085-4560-8cd1-c7e941f96af5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:13.977993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:13.998016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e7ffc-053e-494d-a19a-aaacf6394b4e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:14.005368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1144d5-c085-4560-8cd1-c7e941f96af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-7e1144d5-c085-4560-8cd1-c7e941f96af5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:14.013804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:14.017561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:14.018359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 209/417] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:14.020939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1144d5-c085-4560-8cd1-c7e941f96af5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:14.049727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff230825-471d-4ee7-9196-d7deaf448363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:14.051551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff230825-471d-4ee7-9196-d7deaf448363 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:14.052416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff230825-471d-4ee7-9196-d7deaf448363 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:14.052936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff230825-471d-4ee7-9196-d7deaf448363 None admin] PATH_INFO: `/v3/projects/245371efe76c48ce9eb35e790603face/groups/7360427643e84f4b8de24daaf707fb19/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:14.056500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff230825-471d-4ee7-9196-d7deaf448363 None admin] RBAC: Authorizing `identity:create_grant(project_id=245371efe76c48ce9eb35e790603face, group_id=7360427643e84f4b8de24daaf707fb19, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:14.061172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff230825-471d-4ee7-9196-d7deaf448363 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:14.132122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 209/418] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Jul 24 19:41:13 2022] PUT /identity/v3/projects/245371efe76c48ce9eb35e790603face/groups/7360427643e84f4b8de24daaf707fb19/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:14.162113 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:14.170098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1144d5-c085-4560-8cd1-c7e941f96af5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:14.210008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe092613-5b17-4bd4-9d99-12b1823aef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:14.211179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe092613-5b17-4bd4-9d99-12b1823aef1d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:14.211680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe092613-5b17-4bd4-9d99-12b1823aef1d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:14.212183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe092613-5b17-4bd4-9d99-12b1823aef1d None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:14.213171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe092613-5b17-4bd4-9d99-12b1823aef1d None admin] RBAC: Adding query filter params (name=member) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:14.213724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe092613-5b17-4bd4-9d99-12b1823aef1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:14.215363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe092613-5b17-4bd4-9d99-12b1823aef1d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:14.226242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 210/419] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:41:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:14.231836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff230825-471d-4ee7-9196-d7deaf448363 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:14.389614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff230825-471d-4ee7-9196-d7deaf448363 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:14.441355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbf272-77ba-4a3b-a966-712caff2774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:14.442025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf272-77ba-4a3b-a966-712caff2774a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:14.442315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf272-77ba-4a3b-a966-712caff2774a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:14.442601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf272-77ba-4a3b-a966-712caff2774a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:14.459424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:14.519630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbf272-77ba-4a3b-a966-712caff2774a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:14.521093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbf272-77ba-4a3b-a966-712caff2774a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:14.639123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 210/420] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:14.644786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe092613-5b17-4bd4-9d99-12b1823aef1d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:14.697528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e170a4-b2c1-4776-b141-976ed0ee9658 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:14.697847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e170a4-b2c1-4776-b141-976ed0ee9658 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:14.698219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e170a4-b2c1-4776-b141-976ed0ee9658 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:14.699596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 211/421] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:41:14.704845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b07644-c067-4f94-b8dc-99e78d774a08 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:14.705134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b07644-c067-4f94-b8dc-99e78d774a08 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:14.705414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b07644-c067-4f94-b8dc-99e78d774a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:14.723547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-10b07644-c067-4f94-b8dc-99e78d774a08 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:14.724776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-10b07644-c067-4f94-b8dc-99e78d774a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:14.785534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:14.817017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 212/422] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:14.845220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b07644-c067-4f94-b8dc-99e78d774a08 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:14.863005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe092613-5b17-4bd4-9d99-12b1823aef1d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:14.919146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee9e1e-cc86-479d-b5bf-d24e7e30fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:14.920034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee9e1e-cc86-479d-b5bf-d24e7e30fa2e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:14.920324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee9e1e-cc86-479d-b5bf-d24e7e30fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:14.920598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee9e1e-cc86-479d-b5bf-d24e7e30fa2e None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:14.925420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee9e1e-cc86-479d-b5bf-d24e7e30fa2e None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:14.926903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee9e1e-cc86-479d-b5bf-d24e7e30fa2e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:14.929021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 211/423] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:14 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:14.941701 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee9e1e-cc86-479d-b5bf-d24e7e30fa2e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:14.958784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b07644-c067-4f94-b8dc-99e78d774a08 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:14.993379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:14.996539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679824f1-e237-4f14-9d37-0d83e5b86124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:14.997412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679824f1-e237-4f14-9d37-0d83e5b86124 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:14.997701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679824f1-e237-4f14-9d37-0d83e5b86124 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:14.997980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679824f1-e237-4f14-9d37-0d83e5b86124 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.009507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679824f1-e237-4f14-9d37-0d83e5b86124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-679824f1-e237-4f14-9d37-0d83e5b86124 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:15.017897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:15.018800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:15.019591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 213/424] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:41:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:15.025294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679824f1-e237-4f14-9d37-0d83e5b86124 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:15.069201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee9e1e-cc86-479d-b5bf-d24e7e30fa2e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:15.086366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:15.114424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd09d9b-01ee-4abf-8c4a-48f84491e30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:15.115316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd09d9b-01ee-4abf-8c4a-48f84491e30e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:15.115603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd09d9b-01ee-4abf-8c4a-48f84491e30e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:15.115882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd09d9b-01ee-4abf-8c4a-48f84491e30e None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.118794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd09d9b-01ee-4abf-8c4a-48f84491e30e None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:15.122066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd09d9b-01ee-4abf-8c4a-48f84491e30e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:15.123849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 212/425] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:14 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:15.132395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd09d9b-01ee-4abf-8c4a-48f84491e30e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:15.228658 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679824f1-e237-4f14-9d37-0d83e5b86124 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:15.245447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:15.253321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd09d9b-01ee-4abf-8c4a-48f84491e30e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:15.275189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83395361-3409-406a-b43a-8deffe6e4284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:15.275811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395361-3409-406a-b43a-8deffe6e4284 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:15.276170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395361-3409-406a-b43a-8deffe6e4284 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:15.276576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395361-3409-406a-b43a-8deffe6e4284 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.277659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395361-3409-406a-b43a-8deffe6e4284 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:15.278044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395361-3409-406a-b43a-8deffe6e4284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:15.279594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395361-3409-406a-b43a-8deffe6e4284 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:15.289394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 214/426] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:15.293433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc201e8-4a5e-4703-9869-915e970d693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:15.294450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc201e8-4a5e-4703-9869-915e970d693f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:15.294937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc201e8-4a5e-4703-9869-915e970d693f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:15.295431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc201e8-4a5e-4703-9869-915e970d693f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.296499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27e557-afee-4f09-887d-6ada059c53c0 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:15.296866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27e557-afee-4f09-887d-6ada059c53c0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:15.297160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27e557-afee-4f09-887d-6ada059c53c0 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.297525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc201e8-4a5e-4703-9869-915e970d693f None admin] RBAC: Adding query filter params (group.id=25f360154b9b475590ae79e7b37e0959, role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:15.298039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc201e8-4a5e-4703-9869-915e970d693f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:15.298920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 215/427] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:15.299718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27e557-afee-4f09-887d-6ada059c53c0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:15.300343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc201e8-4a5e-4703-9869-915e970d693f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:15.308459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 213/428] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:15 2022] GET /identity/v3/role_assignments?group.id=25f360154b9b475590ae79e7b37e0959&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:15.310282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cf823-f0b4-4d6c-b9ea-df34f68d22f0 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:15.310815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cf823-f0b4-4d6c-b9ea-df34f68d22f0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:15.311331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cf823-f0b4-4d6c-b9ea-df34f68d22f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.329288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-904cf823-f0b4-4d6c-b9ea-df34f68d22f0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:15.330802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-904cf823-f0b4-4d6c-b9ea-df34f68d22f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:15.420504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 214/429] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:15.468692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cf823-f0b4-4d6c-b9ea-df34f68d22f0 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:15.502518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:15.516169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27e557-afee-4f09-887d-6ada059c53c0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:15.563372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5ac1f-be6e-4699-8436-a1c1d4437510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:15.564451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5ac1f-be6e-4699-8436-a1c1d4437510 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:15.564881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5ac1f-be6e-4699-8436-a1c1d4437510 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:15.565259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5ac1f-be6e-4699-8436-a1c1d4437510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.634163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:15.638005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cf823-f0b4-4d6c-b9ea-df34f68d22f0 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:15.663869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5ac1f-be6e-4699-8436-a1c1d4437510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:15.668428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5ac1f-be6e-4699-8436-a1c1d4437510 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:15.708213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dc091-16d7-4d33-a2b4-837001d91ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:15.709308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dc091-16d7-4d33-a2b4-837001d91ab7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:15.709737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dc091-16d7-4d33-a2b4-837001d91ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:15.710162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dc091-16d7-4d33-a2b4-837001d91ab7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.726291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1dc091-16d7-4d33-a2b4-837001d91ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-4f1dc091-16d7-4d33-a2b4-837001d91ab7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:15.735158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:15.736057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 24 19:41:15.736916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:15.739712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 215/430] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:15.745239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dc091-16d7-4d33-a2b4-837001d91ab7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:15.809725 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:15.816807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 216/431] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:15.822638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5ac1f-be6e-4699-8436-a1c1d4437510 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:15.872549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dc091-16d7-4d33-a2b4-837001d91ab7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:15.908554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734db1b-5808-4da8-bc6b-2d6a4cc2478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:15.909200 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734db1b-5808-4da8-bc6b-2d6a4cc2478d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:15.909485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734db1b-5808-4da8-bc6b-2d6a4cc2478d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:15.909761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734db1b-5808-4da8-bc6b-2d6a4cc2478d None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.910516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734db1b-5808-4da8-bc6b-2d6a4cc2478d None admin] RBAC: Adding query filter params (name=member) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:15.910819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734db1b-5808-4da8-bc6b-2d6a4cc2478d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:15.913311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734db1b-5808-4da8-bc6b-2d6a4cc2478d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:15.925813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 216/432] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:41:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:15.931281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734db1b-5808-4da8-bc6b-2d6a4cc2478d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:15.942460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5ac1f-be6e-4699-8436-a1c1d4437510 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:15.945169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:15.982065 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e174a-c413-4bf9-8865-8fa107c1d7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:15.983374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e174a-c413-4bf9-8865-8fa107c1d7df None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:15.983960 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e174a-c413-4bf9-8865-8fa107c1d7df None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:15.984442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e174a-c413-4bf9-8865-8fa107c1d7df None admin] PATH_INFO: `/v3/projects/04a4d4b6451449af851d7f25e1f9c6be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:15.986058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e174a-c413-4bf9-8865-8fa107c1d7df None admin] RBAC: Authorizing `identity:get_project(project_id=04a4d4b6451449af851d7f25e1f9c6be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:15.987761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e174a-c413-4bf9-8865-8fa107c1d7df None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:15.990060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 217/433] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:15 2022] GET /identity/v3/projects/04a4d4b6451449af851d7f25e1f9c6be => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:15.996146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e174a-c413-4bf9-8865-8fa107c1d7df None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:16.058093 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734db1b-5808-4da8-bc6b-2d6a4cc2478d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:16.076730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:16.104977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1122782-c192-4a3d-a133-23914b5d74fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:16.105644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1122782-c192-4a3d-a133-23914b5d74fb None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.105931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1122782-c192-4a3d-a133-23914b5d74fb None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.106214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1122782-c192-4a3d-a133-23914b5d74fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.133563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e174a-c413-4bf9-8865-8fa107c1d7df None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:16.190914 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e9c9-6824-41e2-a2c1-06e601fab2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:16.191836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e9c9-6824-41e2-a2c1-06e601fab2e8 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.192222 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e9c9-6824-41e2-a2c1-06e601fab2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.192507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e9c9-6824-41e2-a2c1-06e601fab2e8 None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.193813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017e9c9-6824-41e2-a2c1-06e601fab2e8 None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:16.195514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017e9c9-6824-41e2-a2c1-06e601fab2e8 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:16.198708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 218/434] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:15 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:16.206289 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e9c9-6824-41e2-a2c1-06e601fab2e8 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:16.207432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1122782-c192-4a3d-a133-23914b5d74fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:16.208961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1122782-c192-4a3d-a133-23914b5d74fb None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:16.221668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:16.315490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 217/435] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:16.324850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1122782-c192-4a3d-a133-23914b5d74fb None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:16.328313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e9c9-6824-41e2-a2c1-06e601fab2e8 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:16.360517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:16.366768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5716a6-730a-46de-8255-702230315e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:16.367430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5716a6-730a-46de-8255-702230315e70 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.367717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5716a6-730a-46de-8255-702230315e70 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.367993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5716a6-730a-46de-8255-702230315e70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.368853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5716a6-730a-46de-8255-702230315e70 None admin] RBAC: Adding query filter params (group.id=25f360154b9b475590ae79e7b37e0959, role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=04a4d4b6451449af851d7f25e1f9c6be) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:16.369169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5716a6-730a-46de-8255-702230315e70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:16.371148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5716a6-730a-46de-8255-702230315e70 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:16.377466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 219/436] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:16 2022] GET /identity/v3/role_assignments?group.id=25f360154b9b475590ae79e7b37e0959&scope.project.id=04a4d4b6451449af851d7f25e1f9c6be&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:16.380798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f1867-1a6b-4ac6-b7a8-01dea229244f None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.380798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f1867-1a6b-4ac6-b7a8-01dea229244f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.380798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f1867-1a6b-4ac6-b7a8-01dea229244f None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.380798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 220/437] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:16.388242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94c262-c5aa-4b8d-855a-9d1317629289 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.388575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94c262-c5aa-4b8d-855a-9d1317629289 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.389308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94c262-c5aa-4b8d-855a-9d1317629289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.413491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-7b94c262-c5aa-4b8d-855a-9d1317629289 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:16.414723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7b94c262-c5aa-4b8d-855a-9d1317629289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:16.468236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1122782-c192-4a3d-a133-23914b5d74fb None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:16.487292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:16.501053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 221/438] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:16.517302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfe5a9-6a36-45dd-a46f-ca45837c08ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:16.518380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfe5a9-6a36-45dd-a46f-ca45837c08ba None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.518972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfe5a9-6a36-45dd-a46f-ca45837c08ba None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.519250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfe5a9-6a36-45dd-a46f-ca45837c08ba None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.520524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfe5a9-6a36-45dd-a46f-ca45837c08ba None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:16.522088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfe5a9-6a36-45dd-a46f-ca45837c08ba None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:16.524286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 218/439] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:16 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:16.530227 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94c262-c5aa-4b8d-855a-9d1317629289 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:16.530422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfe5a9-6a36-45dd-a46f-ca45837c08ba None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:16.665639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94c262-c5aa-4b8d-855a-9d1317629289 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:16.665851 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfe5a9-6a36-45dd-a46f-ca45837c08ba None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:16.696935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:16.706356 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b77bd6-48a5-4c5b-a4b7-eec0d805a9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:16.709847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc882aed-d4a8-4c00-8b80-2294287038ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:16.709968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b77bd6-48a5-4c5b-a4b7-eec0d805a9b5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.710058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc882aed-d4a8-4c00-8b80-2294287038ab None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.710140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b77bd6-48a5-4c5b-a4b7-eec0d805a9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.710140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b77bd6-48a5-4c5b-a4b7-eec0d805a9b5 None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.710296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc882aed-d4a8-4c00-8b80-2294287038ab None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.710296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc882aed-d4a8-4c00-8b80-2294287038ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.710423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b77bd6-48a5-4c5b-a4b7-eec0d805a9b5 None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:16.710805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b77bd6-48a5-4c5b-a4b7-eec0d805a9b5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:16.712482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 219/440] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:16 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:16.715578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4ec30-8ece-4d85-941e-c236c22f7dfd None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.715866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4ec30-8ece-4d85-941e-c236c22f7dfd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.716148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4ec30-8ece-4d85-941e-c236c22f7dfd None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.717584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 220/441] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:16.721961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4ec30-8ece-4d85-941e-c236c22f7dfd None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:16.723370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc882aed-d4a8-4c00-8b80-2294287038ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-fc882aed-d4a8-4c00-8b80-2294287038ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:16.732403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:16.733419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:16.734242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:16.737195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 222/442] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:41:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:16.738547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90b699-b439-485b-b477-58f302d58a68 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.739300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90b699-b439-485b-b477-58f302d58a68 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.739571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90b699-b439-485b-b477-58f302d58a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.757874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-bc90b699-b439-485b-b477-58f302d58a68 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:16.759069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-bc90b699-b439-485b-b477-58f302d58a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:16.774469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:16.832937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4ec30-8ece-4d85-941e-c236c22f7dfd None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:16.834827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 223/443] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:16.835683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90b699-b439-485b-b477-58f302d58a68 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:16.870718 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98496c9b-916c-4a36-a971-d177231c2f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:16.871379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98496c9b-916c-4a36-a971-d177231c2f16 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.871672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98496c9b-916c-4a36-a971-d177231c2f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.871957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98496c9b-916c-4a36-a971-d177231c2f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.872825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98496c9b-916c-4a36-a971-d177231c2f16 None admin] RBAC: Adding query filter params (group.id=25f360154b9b475590ae79e7b37e0959, role.id=2a20315693ce44d696f86173c0f6eb09, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:16.873148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98496c9b-916c-4a36-a971-d177231c2f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:16.874587 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98496c9b-916c-4a36-a971-d177231c2f16 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:16.892680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 221/444] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:16 2022] GET /identity/v3/role_assignments?group.id=25f360154b9b475590ae79e7b37e0959&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=2a20315693ce44d696f86173c0f6eb09 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:16.892680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98496c9b-916c-4a36-a971-d177231c2f16 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:16.946332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90b699-b439-485b-b477-58f302d58a68 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:16.965485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:16.987081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c962a-7e8a-4497-b8e3-14bfc133aebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:16.987788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c962a-7e8a-4497-b8e3-14bfc133aebc None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:16.988074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c962a-7e8a-4497-b8e3-14bfc133aebc None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:16.988349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c962a-7e8a-4497-b8e3-14bfc133aebc None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:16.989156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c962a-7e8a-4497-b8e3-14bfc133aebc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:16.989467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c962a-7e8a-4497-b8e3-14bfc133aebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:16.990911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c962a-7e8a-4497-b8e3-14bfc133aebc None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:16.999947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 224/445] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:17.005742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c962a-7e8a-4497-b8e3-14bfc133aebc None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:17.007089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98496c9b-916c-4a36-a971-d177231c2f16 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:17.022350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:17.050921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a291d-16d4-4081-9f79-c96902074a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:17.051779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a291d-16d4-4081-9f79-c96902074a6d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:17.052018 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a291d-16d4-4081-9f79-c96902074a6d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:17.057595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a291d-16d4-4081-9f79-c96902074a6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:17.069379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a291d-16d4-4081-9f79-c96902074a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-b92a291d-16d4-4081-9f79-c96902074a6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:17.077873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:17.081134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:17.081924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 222/446] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:17.085461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a291d-16d4-4081-9f79-c96902074a6d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:17.131772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c962a-7e8a-4497-b8e3-14bfc133aebc None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:17.170452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2bad-937e-44df-9716-5b3d0f35c2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:17.171114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2bad-937e-44df-9716-5b3d0f35c2ad None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:17.171395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2bad-937e-44df-9716-5b3d0f35c2ad None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:17.171672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2bad-937e-44df-9716-5b3d0f35c2ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:17.212105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a291d-16d4-4081-9f79-c96902074a6d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:17.223938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:17.248024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc2bad-937e-44df-9716-5b3d0f35c2ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:17.249475 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc2bad-937e-44df-9716-5b3d0f35c2ad None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:17.258296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c076d04-8060-4217-a2df-7cc0e5f7d7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:17.258969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c076d04-8060-4217-a2df-7cc0e5f7d7d1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:17.259307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c076d04-8060-4217-a2df-7cc0e5f7d7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:17.259595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c076d04-8060-4217-a2df-7cc0e5f7d7d1 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:17.260389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c076d04-8060-4217-a2df-7cc0e5f7d7d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:17.261246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c076d04-8060-4217-a2df-7cc0e5f7d7d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:17.262626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c076d04-8060-4217-a2df-7cc0e5f7d7d1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:17.270833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 223/447] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:17.276428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c076d04-8060-4217-a2df-7cc0e5f7d7d1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:17.306884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:17.352377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 225/448] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 24 19:41:17.357975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2bad-937e-44df-9716-5b3d0f35c2ad None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:17.374985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c076d04-8060-4217-a2df-7cc0e5f7d7d1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:17.410725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0cfa9-de92-4496-889e-dcbe1aedbbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:17.411395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0cfa9-de92-4496-889e-dcbe1aedbbcd None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:17.411689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0cfa9-de92-4496-889e-dcbe1aedbbcd None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:17.411973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0cfa9-de92-4496-889e-dcbe1aedbbcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:17.439136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:17.465641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2bad-937e-44df-9716-5b3d0f35c2ad None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:17.486925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0cfa9-de92-4496-889e-dcbe1aedbbcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:17.488388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0cfa9-de92-4496-889e-dcbe1aedbbcd None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:17.499777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154f802-1da0-46eb-a554-fd38e8f1484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:17.500573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154f802-1da0-46eb-a554-fd38e8f1484f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:17.500884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154f802-1da0-46eb-a554-fd38e8f1484f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:17.501158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154f802-1da0-46eb-a554-fd38e8f1484f None admin] PATH_INFO: `/v3/projects/6f531532fbb141bd81ca33ecf527a7cc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:17.502346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154f802-1da0-46eb-a554-fd38e8f1484f None admin] RBAC: Authorizing `identity:get_project(project_id=6f531532fbb141bd81ca33ecf527a7cc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:17.503769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154f802-1da0-46eb-a554-fd38e8f1484f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:17.505486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 226/449] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:17 2022] GET /identity/v3/projects/6f531532fbb141bd81ca33ecf527a7cc => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:17.511385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154f802-1da0-46eb-a554-fd38e8f1484f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:17.518244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:17.586161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 224/450] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 24 19:41:17.592314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0cfa9-de92-4496-889e-dcbe1aedbbcd None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:17.612456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154f802-1da0-46eb-a554-fd38e8f1484f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:17.646325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aec20-10ce-466d-bcad-7293b81f9283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:17.647178 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aec20-10ce-466d-bcad-7293b81f9283 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:17.647461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aec20-10ce-466d-bcad-7293b81f9283 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:17.647732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aec20-10ce-466d-bcad-7293b81f9283 None admin] PATH_INFO: `/v3/groups/25f360154b9b475590ae79e7b37e0959` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:17.648930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aec20-10ce-466d-bcad-7293b81f9283 None admin] RBAC: Authorizing `identity:get_group(group_id=25f360154b9b475590ae79e7b37e0959)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:17.650353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aec20-10ce-466d-bcad-7293b81f9283 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:17.652569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 227/451] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:17 2022] GET /identity/v3/groups/25f360154b9b475590ae79e7b37e0959 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:17.658251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aec20-10ce-466d-bcad-7293b81f9283 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:17.673825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:17.689982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0cfa9-de92-4496-889e-dcbe1aedbbcd None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:17.718101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:17.723928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12782-70d1-4c81-802e-cb8c6fa1c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:17.724861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12782-70d1-4c81-802e-cb8c6fa1c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:17.725149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12782-70d1-4c81-802e-cb8c6fa1c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:17.725421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12782-70d1-4c81-802e-cb8c6fa1c3f4 None admin] PATH_INFO: `/v3/projects/245371efe76c48ce9eb35e790603face` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:17.726594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12782-70d1-4c81-802e-cb8c6fa1c3f4 None admin] RBAC: Authorizing `identity:get_project(project_id=245371efe76c48ce9eb35e790603face)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:17.728026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12782-70d1-4c81-802e-cb8c6fa1c3f4 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:17.730334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 225/452] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:41:17 2022] GET /identity/v3/projects/245371efe76c48ce9eb35e790603face => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:17.735672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12782-70d1-4c81-802e-cb8c6fa1c3f4 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:17.761382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aec20-10ce-466d-bcad-7293b81f9283 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:17.795250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772e55c-73ef-4dc4-8321-b45681b0e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:17.795898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772e55c-73ef-4dc4-8321-b45681b0e4fc None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:17.796182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772e55c-73ef-4dc4-8321-b45681b0e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:17.796470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772e55c-73ef-4dc4-8321-b45681b0e4fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:17.797379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772e55c-73ef-4dc4-8321-b45681b0e4fc None admin] RBAC: Adding query filter params (group.id=25f360154b9b475590ae79e7b37e0959, role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=6f531532fbb141bd81ca33ecf527a7cc) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:17.797657 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772e55c-73ef-4dc4-8321-b45681b0e4fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:17.799080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772e55c-73ef-4dc4-8321-b45681b0e4fc None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:17.804824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 228/453] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:17 2022] GET /identity/v3/role_assignments?group.id=25f360154b9b475590ae79e7b37e0959&scope.project.id=6f531532fbb141bd81ca33ecf527a7cc&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:17.838779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12782-70d1-4c81-802e-cb8c6fa1c3f4 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:17.875725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5918cc-80da-4187-9e92-4c84be205d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:17.876606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5918cc-80da-4187-9e92-4c84be205d6f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:17.876920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5918cc-80da-4187-9e92-4c84be205d6f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:17.877198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5918cc-80da-4187-9e92-4c84be205d6f None admin] PATH_INFO: `/v3/groups/7360427643e84f4b8de24daaf707fb19` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:17.878423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5918cc-80da-4187-9e92-4c84be205d6f None admin] RBAC: Authorizing `identity:get_group(group_id=7360427643e84f4b8de24daaf707fb19)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:17.879898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5918cc-80da-4187-9e92-4c84be205d6f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:17.881643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 226/454] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Jul 24 19:41:17 2022] GET /identity/v3/groups/7360427643e84f4b8de24daaf707fb19 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:17.887946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772e55c-73ef-4dc4-8321-b45681b0e4fc None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:17.969970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:17.992690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772e55c-73ef-4dc4-8321-b45681b0e4fc None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:18.028117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181434cc-eba6-4019-b9f9-2fcee4a06552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:18.028743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181434cc-eba6-4019-b9f9-2fcee4a06552 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:18.029032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181434cc-eba6-4019-b9f9-2fcee4a06552 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:18.029317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181434cc-eba6-4019-b9f9-2fcee4a06552 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:18.030156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181434cc-eba6-4019-b9f9-2fcee4a06552 None admin] RBAC: Adding query filter params (group.id=7360427643e84f4b8de24daaf707fb19, role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=245371efe76c48ce9eb35e790603face) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:18.030477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181434cc-eba6-4019-b9f9-2fcee4a06552 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:18.031925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181434cc-eba6-4019-b9f9-2fcee4a06552 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:18.037969 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 229/455] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Jul 24 19:41:17 2022] GET /identity/v3/role_assignments?group.id=7360427643e84f4b8de24daaf707fb19&scope.project.id=245371efe76c48ce9eb35e790603face&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:20.521302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d35aa0-a444-4759-8fa1-202fc1211023 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:20.521800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d35aa0-a444-4759-8fa1-202fc1211023 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:20.521905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d35aa0-a444-4759-8fa1-202fc1211023 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:20.523440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 227/456] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:20.529116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305b611-463d-4d0f-9cc0-1f4aa4fd0ec9 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:20.529511 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305b611-463d-4d0f-9cc0-1f4aa4fd0ec9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:20.529686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305b611-463d-4d0f-9cc0-1f4aa4fd0ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:20.558278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c305b611-463d-4d0f-9cc0-1f4aa4fd0ec9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:20.559478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c305b611-463d-4d0f-9cc0-1f4aa4fd0ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:20.642926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 230/457] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:20.694293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d35aa0-a444-4759-8fa1-202fc1211023 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:20.699845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57128ef-4373-4431-9c0c-ec4914642ebb None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:20.700130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57128ef-4373-4431-9c0c-ec4914642ebb None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:20.700686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57128ef-4373-4431-9c0c-ec4914642ebb None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:20.702531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 231/458] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:20.707688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fb4c-ca40-4b72-b9a7-01908807e0c4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:20.708206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fb4c-ca40-4b72-b9a7-01908807e0c4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:20.708699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fb4c-ca40-4b72-b9a7-01908807e0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:20.726240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a331fb4c-ca40-4b72-b9a7-01908807e0c4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:20.727911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a331fb4c-ca40-4b72-b9a7-01908807e0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:20.810940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 232/459] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:20.812721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571ec27-d515-4560-8e74-33e04f7024eb None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:20.813214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571ec27-d515-4560-8e74-33e04f7024eb None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:20.813715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571ec27-d515-4560-8e74-33e04f7024eb None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:20.815421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 233/460] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:20.820317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9430165-df01-4612-a4f6-634c7c9f50b3 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:20.820961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9430165-df01-4612-a4f6-634c7c9f50b3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:20.821440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9430165-df01-4612-a4f6-634c7c9f50b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:20.832986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:20.842192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:20.849970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c9430165-df01-4612-a4f6-634c7c9f50b3 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:20.851654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c9430165-df01-4612-a4f6-634c7c9f50b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:20.868745 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d35aa0-a444-4759-8fa1-202fc1211023 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:20.913996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192416d-cfec-4f7e-a1f8-b453a452796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:20.915437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192416d-cfec-4f7e-a1f8-b453a452796d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:20.916123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192416d-cfec-4f7e-a1f8-b453a452796d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:20.917218 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192416d-cfec-4f7e-a1f8-b453a452796d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:20.931727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192416d-cfec-4f7e-a1f8-b453a452796d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:20.933699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192416d-cfec-4f7e-a1f8-b453a452796d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:20.944187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 234/461] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:20.945481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9430165-df01-4612-a4f6-634c7c9f50b3 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-8192416d-cfec-4f7e-a1f8-b453a452796d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:20.962490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:20.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:20.964823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:20.964823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:20.964823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:20.964823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:20.970277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 228/462] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:20.971682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192416d-cfec-4f7e-a1f8-b453a452796d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:21.071813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9430165-df01-4612-a4f6-634c7c9f50b3 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:21.080168 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192416d-cfec-4f7e-a1f8-b453a452796d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:21.102463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:21.106422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:21.111699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35586d-699c-4c28-b942-d34cf82426f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:21.112850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35586d-699c-4c28-b942-d34cf82426f9 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.118611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35586d-699c-4c28-b942-d34cf82426f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.118611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35586d-699c-4c28-b942-d34cf82426f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.120007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7303e-063b-4e96-ba03-277398ea25da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:21.121405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7303e-063b-4e96-ba03-277398ea25da None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.121922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7303e-063b-4e96-ba03-277398ea25da None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.122380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7303e-063b-4e96-ba03-277398ea25da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.127826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35586d-699c-4c28-b942-d34cf82426f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:21.129836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35586d-699c-4c28-b942-d34cf82426f9 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:21.135749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7303e-063b-4e96-ba03-277398ea25da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:21.137239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7303e-063b-4e96-ba03-277398ea25da None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-4c35586d-699c-4c28-b942-d34cf82426f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:21.150673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:21.152060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:21.152935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:21.152935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:21.152935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:21.152935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-e8b7303e-063b-4e96-ba03-277398ea25da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:21.155589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:21.156604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:21.163428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:21.163428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:21.163428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:21.163428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:21.163651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 235/463] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:21.163651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35586d-699c-4c28-b942-d34cf82426f9 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:21.163773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 229/464] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:21.164845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7303e-063b-4e96-ba03-277398ea25da None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:21.272515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35586d-699c-4c28-b942-d34cf82426f9 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:21.273408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7303e-063b-4e96-ba03-277398ea25da None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:21.310058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8945a35-3703-4e6c-8d4b-85fc7e790bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:21.310925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f47e9-378d-4bc9-b845-f339454057c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:21.311224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8945a35-3703-4e6c-8d4b-85fc7e790bb0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.311714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8945a35-3703-4e6c-8d4b-85fc7e790bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.311817 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f47e9-378d-4bc9-b845-f339454057c7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.312109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f47e9-378d-4bc9-b845-f339454057c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.312446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8945a35-3703-4e6c-8d4b-85fc7e790bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.312545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f47e9-378d-4bc9-b845-f339454057c7 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.313285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f47e9-378d-4bc9-b845-f339454057c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:21.313681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8945a35-3703-4e6c-8d4b-85fc7e790bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:21.313781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f47e9-378d-4bc9-b845-f339454057c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:21.314296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8945a35-3703-4e6c-8d4b-85fc7e790bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:21.315130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f47e9-378d-4bc9-b845-f339454057c7 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:21.316078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8945a35-3703-4e6c-8d4b-85fc7e790bb0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:21.329635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 230/465] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:21.330777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8945a35-3703-4e6c-8d4b-85fc7e790bb0 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:21.333293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 236/466] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:21.334303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f47e9-378d-4bc9-b845-f339454057c7 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:21.365930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:21.385909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:21.444951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8945a35-3703-4e6c-8d4b-85fc7e790bb0 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:21.455669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f47e9-378d-4bc9-b845-f339454057c7 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:21.482439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378298d-169b-4a93-8c45-3396a896ef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:21.483106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378298d-169b-4a93-8c45-3396a896ef90 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.483396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378298d-169b-4a93-8c45-3396a896ef90 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.483683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378298d-169b-4a93-8c45-3396a896ef90 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.484409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378298d-169b-4a93-8c45-3396a896ef90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:21.484781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378298d-169b-4a93-8c45-3396a896ef90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:21.486163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378298d-169b-4a93-8c45-3396a896ef90 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:21.491734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6bbd-221b-4b31-95d8-b5d56c2d76e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:21.492623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6bbd-221b-4b31-95d8-b5d56c2d76e9 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.492938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6bbd-221b-4b31-95d8-b5d56c2d76e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.493223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6bbd-221b-4b31-95d8-b5d56c2d76e9 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.493946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6bbd-221b-4b31-95d8-b5d56c2d76e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:21.495939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6bbd-221b-4b31-95d8-b5d56c2d76e9 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:21.503277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 231/467] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:21.503277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378298d-169b-4a93-8c45-3396a896ef90 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:21.589212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 237/468] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:41:21 2022] POST /identity/v3/users => generated 246 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:21.590691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305e5d0-426b-4a2d-8bad-ba9b7267797b None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.591004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305e5d0-426b-4a2d-8bad-ba9b7267797b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.591277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305e5d0-426b-4a2d-8bad-ba9b7267797b None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.597010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 238/469] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:21.597010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305e5d0-426b-4a2d-8bad-ba9b7267797b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:21.601005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:21.612274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378298d-169b-4a93-8c45-3396a896ef90 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:21.650603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95206271-3cbe-47cd-829d-72bc4c81a751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:21.651821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95206271-3cbe-47cd-829d-72bc4c81a751 None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.652107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95206271-3cbe-47cd-829d-72bc4c81a751 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.652382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95206271-3cbe-47cd-829d-72bc4c81a751 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.653130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95206271-3cbe-47cd-829d-72bc4c81a751 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:21.654599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95206271-3cbe-47cd-829d-72bc4c81a751 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:21.738186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 232/470] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:41:21 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:21.740495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59589b1-c072-4ccc-b844-fee15666be27 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.740823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59589b1-c072-4ccc-b844-fee15666be27 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.741107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59589b1-c072-4ccc-b844-fee15666be27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.763393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:21.769220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305e5d0-426b-4a2d-8bad-ba9b7267797b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:21.769801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e59589b1-c072-4ccc-b844-fee15666be27 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:21.771393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e59589b1-c072-4ccc-b844-fee15666be27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:21.810080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b084f6-8289-4e9b-bae2-580f27d4dee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:21.810771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b084f6-8289-4e9b-bae2-580f27d4dee6 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:21.811917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b084f6-8289-4e9b-bae2-580f27d4dee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:21.811917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b084f6-8289-4e9b-bae2-580f27d4dee6 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:21.812090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b084f6-8289-4e9b-bae2-580f27d4dee6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:21.813596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b084f6-8289-4e9b-bae2-580f27d4dee6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:21.859136 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 233/471] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:21.884047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59589b1-c072-4ccc-b844-fee15666be27 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:21.896957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 239/472] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:41:21 2022] POST /identity/v3/users => generated 248 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:21.976729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:22.002598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59589b1-c072-4ccc-b844-fee15666be27 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:22.014600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbe73e-6621-4f65-ade9-2f0fe8873540 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:22.014902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbe73e-6621-4f65-ade9-2f0fe8873540 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:22.015162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbe73e-6621-4f65-ade9-2f0fe8873540 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:22.016754 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 240/473] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:41:22.023213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c47b2-621f-47c2-82f9-3e04029ba530 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:22.023498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c47b2-621f-47c2-82f9-3e04029ba530 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:22.023788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c47b2-621f-47c2-82f9-3e04029ba530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:22.043132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6a7c47b2-621f-47c2-82f9-3e04029ba530 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:22.044581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6a7c47b2-621f-47c2-82f9-3e04029ba530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:22.049575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edcca7-5add-4810-b06d-c28ce1a2adcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:22.050589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edcca7-5add-4810-b06d-c28ce1a2adcc None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:22.050959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edcca7-5add-4810-b06d-c28ce1a2adcc None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:22.051337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edcca7-5add-4810-b06d-c28ce1a2adcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:22.063365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edcca7-5add-4810-b06d-c28ce1a2adcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:22.065133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edcca7-5add-4810-b06d-c28ce1a2adcc None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-e9edcca7-5add-4810-b06d-c28ce1a2adcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:22.085647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:22.086532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:22.087363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:22.087363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:22.087363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:22.087363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:22.087363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 234/474] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:22.094525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edcca7-5add-4810-b06d-c28ce1a2adcc None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:22.127789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 241/475] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:22.152196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c47b2-621f-47c2-82f9-3e04029ba530 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:22.218621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edcca7-5add-4810-b06d-c28ce1a2adcc None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:22.255533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:22.261139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:22.267570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdfef5-fc57-49c2-ad59-3dd1fb2e3f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:22.268604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdfef5-fc57-49c2-ad59-3dd1fb2e3f92 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:22.269055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdfef5-fc57-49c2-ad59-3dd1fb2e3f92 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:22.269472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdfef5-fc57-49c2-ad59-3dd1fb2e3f92 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:22.270976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdfef5-fc57-49c2-ad59-3dd1fb2e3f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:22.271429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdfef5-fc57-49c2-ad59-3dd1fb2e3f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:22.272676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c47b2-621f-47c2-82f9-3e04029ba530 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:22.273234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdfef5-fc57-49c2-ad59-3dd1fb2e3f92 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:22.290376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 235/476] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:22.297676 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdfef5-fc57-49c2-ad59-3dd1fb2e3f92 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:22.320209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:22.323611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:22.323611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:22.323611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:22.324153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:22.325794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:22.368512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.sql.core [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 24 19:41:22.368512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 24 19:41:22.368512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [parameters: {'id': '2909a25245d7466a8d099cb370c20a1b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 24 19:41:22.368512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86806) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application result = self._query(query) Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application conn.query(q) Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application result.read() Jul 24 19:41:22.390711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return ref.to_dict() Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 24 19:41:22.391732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application next(self.gen) Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application yield resource Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application next(self.gen) Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.session.rollback() Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.force_reraise() Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise self.value Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application session.commit() Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 24 19:41:22.392681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._prepare_impl() Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.session.flush() Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._flush(objects) Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application compat.raise_( Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application flush_context.execute() Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rec.execute(self) Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 24 19:41:22.393554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 24 19:41:22.394442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application result = self._query(query) Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application conn.query(q) Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application result.read() Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 24 19:41:22.395257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [parameters: {'id': '2909a25245d7466a8d099cb370c20a1b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:22.396100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 24 19:41:22.397084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:22.402272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 242/477] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Jul 24 19:41:22 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:22.409920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:22.473769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdfef5-fc57-49c2-ad59-3dd1fb2e3f92 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:22.559952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16314c2-b66d-4c0d-9d6e-706ad969a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:22.560630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16314c2-b66d-4c0d-9d6e-706ad969a58b None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:22.560933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16314c2-b66d-4c0d-9d6e-706ad969a58b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:22.561213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16314c2-b66d-4c0d-9d6e-706ad969a58b None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:22.561976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16314c2-b66d-4c0d-9d6e-706ad969a58b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:22.563281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1442a15-59a8-4b4c-8d01-cde87d8232a3 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:22.563440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16314c2-b66d-4c0d-9d6e-706ad969a58b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:22.600082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c83bb-f16c-4324-a242-317d3822f815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:22.601580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c83bb-f16c-4324-a242-317d3822f815 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:22.602087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c83bb-f16c-4324-a242-317d3822f815 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:22.602569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c83bb-f16c-4324-a242-317d3822f815 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:22.613737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c83bb-f16c-4324-a242-317d3822f815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-ee4c83bb-f16c-4324-a242-317d3822f815 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:22.622805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:22.623779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 24 19:41:22.624566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:22.629276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 243/478] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:41:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:22.632060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c83bb-f16c-4324-a242-317d3822f815 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:22.646893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:22.657229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 236/479] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:41:22 2022] POST /identity/v3/users => generated 249 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:22.751263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c83bb-f16c-4324-a242-317d3822f815 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:22.789031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed10f22-399b-4370-9c2c-6ef7ca437935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:22.789726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed10f22-399b-4370-9c2c-6ef7ca437935 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:22.790013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed10f22-399b-4370-9c2c-6ef7ca437935 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:22.790295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed10f22-399b-4370-9c2c-6ef7ca437935 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:22.791121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed10f22-399b-4370-9c2c-6ef7ca437935 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:22.791536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed10f22-399b-4370-9c2c-6ef7ca437935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:22.793236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed10f22-399b-4370-9c2c-6ef7ca437935 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:22.802955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 244/480] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:24.237187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370cbda-f1b6-46ae-b994-b5091fdd66d1 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:24.237720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370cbda-f1b6-46ae-b994-b5091fdd66d1 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:24.237795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370cbda-f1b6-46ae-b994-b5091fdd66d1 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:24.239376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 237/481] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:24.253076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6570f-5ddc-48df-bcbc-954cb73cc6b2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:24.253580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6570f-5ddc-48df-bcbc-954cb73cc6b2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:24.253653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6570f-5ddc-48df-bcbc-954cb73cc6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:24.281667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e2b6570f-5ddc-48df-bcbc-954cb73cc6b2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:24.282865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e2b6570f-5ddc-48df-bcbc-954cb73cc6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:24.297017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6767c-7f53-4f9a-baba-c88900f3318d None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:24.297355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6767c-7f53-4f9a-baba-c88900f3318d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:24.297642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6767c-7f53-4f9a-baba-c88900f3318d None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:24.299242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 238/482] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:41:24.304771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0381-afc5-4a85-95f0-c72f0678766d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:24.305064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0381-afc5-4a85-95f0-c72f0678766d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:24.305347 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0381-afc5-4a85-95f0-c72f0678766d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:24.334808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-6f4c0381-afc5-4a85-95f0-c72f0678766d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:24.336021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6f4c0381-afc5-4a85-95f0-c72f0678766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:24.415013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 245/483] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:24.423811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:24.452200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6570f-5ddc-48df-bcbc-954cb73cc6b2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:24.454513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:24.467638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 239/484] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:24.492121 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba191a-c9b6-4ea9-a003-ab7a325976c6 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:24.492400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba191a-c9b6-4ea9-a003-ab7a325976c6 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:24.492609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba191a-c9b6-4ea9-a003-ab7a325976c6 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:24.494005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 240/485] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:24.501795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea17cdd-2875-49b0-b4d2-6bea732dbd43 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:24.502685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea17cdd-2875-49b0-b4d2-6bea732dbd43 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:24.503187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea17cdd-2875-49b0-b4d2-6bea732dbd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:24.526970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-8ea17cdd-2875-49b0-b4d2-6bea732dbd43 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:24.528680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8ea17cdd-2875-49b0-b4d2-6bea732dbd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:24.571243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6570f-5ddc-48df-bcbc-954cb73cc6b2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:24.611146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41bb0e-0e3c-4c4a-8a21-8d823169d751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:24.612234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41bb0e-0e3c-4c4a-8a21-8d823169d751 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:24.612525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41bb0e-0e3c-4c4a-8a21-8d823169d751 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:24.612840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41bb0e-0e3c-4c4a-8a21-8d823169d751 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:24.620046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 241/486] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:24.621360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea17cdd-2875-49b0-b4d2-6bea732dbd43 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:24.629363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41bb0e-0e3c-4c4a-8a21-8d823169d751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-5e41bb0e-0e3c-4c4a-8a21-8d823169d751 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:24.647851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:24.648876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:24.649677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 246/487] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:24.650941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41bb0e-0e3c-4c4a-8a21-8d823169d751 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:24.741107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:24.766923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea17cdd-2875-49b0-b4d2-6bea732dbd43 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:24.777522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:24.779392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41bb0e-0e3c-4c4a-8a21-8d823169d751 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:24.812628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02d639-75d2-4d45-b112-cc2b60a77b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:24.814783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02d639-75d2-4d45-b112-cc2b60a77b62 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:24.815666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02d639-75d2-4d45-b112-cc2b60a77b62 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:24.816154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02d639-75d2-4d45-b112-cc2b60a77b62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:24.819898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2bec5-07aa-4e98-9eea-98cbf537b7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:24.820718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2bec5-07aa-4e98-9eea-98cbf537b7ac None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:24.821003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2bec5-07aa-4e98-9eea-98cbf537b7ac None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:24.821286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2bec5-07aa-4e98-9eea-98cbf537b7ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:24.829777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02d639-75d2-4d45-b112-cc2b60a77b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:24.833607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2bec5-07aa-4e98-9eea-98cbf537b7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-0b02d639-75d2-4d45-b112-cc2b60a77b62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:24.838964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:24.839869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:24.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-0de2bec5-07aa-4e98-9eea-98cbf537b7ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:24.841858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:24.842685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:24.843487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:24.850542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 247/489] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:24.850542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2bec5-07aa-4e98-9eea-98cbf537b7ac None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:24.850859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 242/489] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:24.851964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02d639-75d2-4d45-b112-cc2b60a77b62 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:24.969064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02d639-75d2-4d45-b112-cc2b60a77b62 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:24.977334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2bec5-07aa-4e98-9eea-98cbf537b7ac None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.007716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:25.012621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c179e-4a64-4050-824c-e83af049a1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.013322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c179e-4a64-4050-824c-e83af049a1a0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.013609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c179e-4a64-4050-824c-e83af049a1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.013888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c179e-4a64-4050-824c-e83af049a1a0 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.014694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c179e-4a64-4050-824c-e83af049a1a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:25.015002 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c179e-4a64-4050-824c-e83af049a1a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.016393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c179e-4a64-4050-824c-e83af049a1a0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.027183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 243/490] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:25.027183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c179e-4a64-4050-824c-e83af049a1a0 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.034877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe334bd-79e6-4ed7-8a35-f1afa42b927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.035813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe334bd-79e6-4ed7-8a35-f1afa42b927a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.036297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe334bd-79e6-4ed7-8a35-f1afa42b927a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.036775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe334bd-79e6-4ed7-8a35-f1afa42b927a None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.037700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe334bd-79e6-4ed7-8a35-f1afa42b927a None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:25.038177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe334bd-79e6-4ed7-8a35-f1afa42b927a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.040955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe334bd-79e6-4ed7-8a35-f1afa42b927a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.053329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 248/491] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:25.054379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe334bd-79e6-4ed7-8a35-f1afa42b927a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.067070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:25.149598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c179e-4a64-4050-824c-e83af049a1a0 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.186338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe334bd-79e6-4ed7-8a35-f1afa42b927a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.188924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c3969-d6d3-4440-aabf-37a77298b961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.190210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c3969-d6d3-4440-aabf-37a77298b961 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.190210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c3969-d6d3-4440-aabf-37a77298b961 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.190379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c3969-d6d3-4440-aabf-37a77298b961 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.191080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c3969-d6d3-4440-aabf-37a77298b961 None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:25.191392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c3969-d6d3-4440-aabf-37a77298b961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.192818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c3969-d6d3-4440-aabf-37a77298b961 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.203360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 244/492] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:25.204477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c3969-d6d3-4440-aabf-37a77298b961 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.231430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d2905-fa22-4a73-ab86-8cb419daa88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.232437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d2905-fa22-4a73-ab86-8cb419daa88a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.233226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d2905-fa22-4a73-ab86-8cb419daa88a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.233226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d2905-fa22-4a73-ab86-8cb419daa88a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.245663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d2905-fa22-4a73-ab86-8cb419daa88a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.247255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d2905-fa22-4a73-ab86-8cb419daa88a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-a75d2905-fa22-4a73-ab86-8cb419daa88a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:25.266635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:25.268029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:25.268919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:25.268919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:25.268919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:25.268919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:25.268919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 249/493] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:25.268919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d2905-fa22-4a73-ab86-8cb419daa88a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.308561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:25.314049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:25.326677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c3969-d6d3-4440-aabf-37a77298b961 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.361060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832d260-3091-4cf1-ad27-6972f8d09ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.361850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832d260-3091-4cf1-ad27-6972f8d09ccd None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.362132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832d260-3091-4cf1-ad27-6972f8d09ccd None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.362405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832d260-3091-4cf1-ad27-6972f8d09ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.373600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832d260-3091-4cf1-ad27-6972f8d09ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.375044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832d260-3091-4cf1-ad27-6972f8d09ccd None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-3832d260-3091-4cf1-ad27-6972f8d09ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:25.393713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:25.397165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:25.398004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:25.398004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:25.398004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:25.398004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:25.398004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 245/494] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:25.398004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21a29a-6bf6-44f0-9eec-1c186f7d0e3b None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.398004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21a29a-6bf6-44f0-9eec-1c186f7d0e3b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.398004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21a29a-6bf6-44f0-9eec-1c186f7d0e3b None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.398904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 246/495] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:25.399402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21a29a-6bf6-44f0-9eec-1c186f7d0e3b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.422922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d2905-fa22-4a73-ab86-8cb419daa88a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.459361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c8bd4-1b86-47cb-a016-a483281f1c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.460152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c8bd4-1b86-47cb-a016-a483281f1c42 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.460438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c8bd4-1b86-47cb-a016-a483281f1c42 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.460742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c8bd4-1b86-47cb-a016-a483281f1c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.472477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c8bd4-1b86-47cb-a016-a483281f1c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.473960 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c8bd4-1b86-47cb-a016-a483281f1c42 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-830c8bd4-1b86-47cb-a016-a483281f1c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:25.492370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:25.493287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:25.494126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:25.494126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:25.494126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:25.494126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:25.494126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 250/496] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:25.494861 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c8bd4-1b86-47cb-a016-a483281f1c42 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.521630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21a29a-6bf6-44f0-9eec-1c186f7d0e3b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.555919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb89cb-bd14-4603-9e3c-f02ceb9b73aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.556577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb89cb-bd14-4603-9e3c-f02ceb9b73aa None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.556915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb89cb-bd14-4603-9e3c-f02ceb9b73aa None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.557197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb89cb-bd14-4603-9e3c-f02ceb9b73aa None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.557925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb89cb-bd14-4603-9e3c-f02ceb9b73aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:25.558240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb89cb-bd14-4603-9e3c-f02ceb9b73aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.559662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb89cb-bd14-4603-9e3c-f02ceb9b73aa None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.570632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 247/497] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:25.571835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38405357-1415-40e3-8543-82f05f199518 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.572129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38405357-1415-40e3-8543-82f05f199518 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.572410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38405357-1415-40e3-8543-82f05f199518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.577343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:25.585955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:25.589050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-38405357-1415-40e3-8543-82f05f199518 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:25.590357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-38405357-1415-40e3-8543-82f05f199518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:25.598023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c8bd4-1b86-47cb-a016-a483281f1c42 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.632609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec06f02-465c-47bb-a248-8a050513b405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.633300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec06f02-465c-47bb-a248-8a050513b405 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.633610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec06f02-465c-47bb-a248-8a050513b405 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.633889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec06f02-465c-47bb-a248-8a050513b405 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.634691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec06f02-465c-47bb-a248-8a050513b405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:25.635015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec06f02-465c-47bb-a248-8a050513b405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.636418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec06f02-465c-47bb-a248-8a050513b405 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.648880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 251/498] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:25.649506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec06f02-465c-47bb-a248-8a050513b405 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.658931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 248/499] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:25.660308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d62469-c5cf-4f72-92c4-1321d28cc66c None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.660608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d62469-c5cf-4f72-92c4-1321d28cc66c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.660915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d62469-c5cf-4f72-92c4-1321d28cc66c None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.662168 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 249/500] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:25.663338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d62469-c5cf-4f72-92c4-1321d28cc66c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.752767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec06f02-465c-47bb-a248-8a050513b405 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.764124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d62469-c5cf-4f72-92c4-1321d28cc66c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.786152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f16a6-5dfe-48a3-88a5-e6079ba2874e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.786807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f16a6-5dfe-48a3-88a5-e6079ba2874e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.787086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f16a6-5dfe-48a3-88a5-e6079ba2874e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.787369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f16a6-5dfe-48a3-88a5-e6079ba2874e None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.788076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f16a6-5dfe-48a3-88a5-e6079ba2874e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:25.788381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f16a6-5dfe-48a3-88a5-e6079ba2874e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.789774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f16a6-5dfe-48a3-88a5-e6079ba2874e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.793511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:25.798388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a0621-a209-4d13-92fc-b37fa0396975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.799177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a0621-a209-4d13-92fc-b37fa0396975 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.799469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a0621-a209-4d13-92fc-b37fa0396975 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.799749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a0621-a209-4d13-92fc-b37fa0396975 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.805842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 252/501] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:25.805842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f16a6-5dfe-48a3-88a5-e6079ba2874e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.811100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a0621-a209-4d13-92fc-b37fa0396975 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.811100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a0621-a209-4d13-92fc-b37fa0396975 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.816109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-6d6a0621-a209-4d13-92fc-b37fa0396975 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:25.826988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:25.833380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 250/502] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d430215-46e4-4e07-9516-ce4b615f227c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.834544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d430215-46e4-4e07-9516-ce4b615f227c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.835462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d430215-46e4-4e07-9516-ce4b615f227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.850306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-5d430215-46e4-4e07-9516-ce4b615f227c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:25.851490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5d430215-46e4-4e07-9516-ce4b615f227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:25.908359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f16a6-5dfe-48a3-88a5-e6079ba2874e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:25.917284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 251/503] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:25.917958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d430215-46e4-4e07-9516-ce4b615f227c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:25.942230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05482391-181a-4a35-9df1-4a6f62def5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:25.943091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05482391-181a-4a35-9df1-4a6f62def5c2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:25.943442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05482391-181a-4a35-9df1-4a6f62def5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:25.943804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05482391-181a-4a35-9df1-4a6f62def5c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:25.951906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05482391-181a-4a35-9df1-4a6f62def5c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:25.953385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05482391-181a-4a35-9df1-4a6f62def5c2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-05482391-181a-4a35-9df1-4a6f62def5c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:25.969165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:25.970248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:25.971170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 253/504] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:25.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05482391-181a-4a35-9df1-4a6f62def5c2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.019193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d430215-46e4-4e07-9516-ce4b615f227c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.038194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:26.043193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:26.054287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722524a-988b-4660-8346-38fb869b8375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.055080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722524a-988b-4660-8346-38fb869b8375 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.057568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722524a-988b-4660-8346-38fb869b8375 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.057568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722524a-988b-4660-8346-38fb869b8375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.065629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722524a-988b-4660-8346-38fb869b8375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.067036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722524a-988b-4660-8346-38fb869b8375 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.071651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05482391-181a-4a35-9df1-4a6f62def5c2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-7722524a-988b-4660-8346-38fb869b8375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:26.084960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:26.087236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:26.088089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:26.088089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:26.088089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:26.088089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:26.088089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 252/505] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:26.088089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722524a-988b-4660-8346-38fb869b8375 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.107335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6eb235-153a-4397-9d44-8f153ecf4bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.108130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6eb235-153a-4397-9d44-8f153ecf4bcd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.108567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6eb235-153a-4397-9d44-8f153ecf4bcd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.108879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6eb235-153a-4397-9d44-8f153ecf4bcd None admin] PATH_INFO: `/v3/users/glance` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.118294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6eb235-153a-4397-9d44-8f153ecf4bcd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.121389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6eb235-153a-4397-9d44-8f153ecf4bcd None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-ae6eb235-153a-4397-9d44-8f153ecf4bcd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:26.140033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:26.142024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 254/506] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:26.144055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6eb235-153a-4397-9d44-8f153ecf4bcd None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.195186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722524a-988b-4660-8346-38fb869b8375 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.228426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151731f1-3009-44e1-b56a-6905c6d70561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.229143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151731f1-3009-44e1-b56a-6905c6d70561 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.229426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151731f1-3009-44e1-b56a-6905c6d70561 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.229699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151731f1-3009-44e1-b56a-6905c6d70561 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.230526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151731f1-3009-44e1-b56a-6905c6d70561 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:26.230837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151731f1-3009-44e1-b56a-6905c6d70561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.232271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151731f1-3009-44e1-b56a-6905c6d70561 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.249165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6eb235-153a-4397-9d44-8f153ecf4bcd None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.268451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 253/507] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Jul 24 19:41:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:26.269313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151731f1-3009-44e1-b56a-6905c6d70561 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.275789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:26.283326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324e9ef-e009-4d1c-8b0e-81b402213205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.284125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324e9ef-e009-4d1c-8b0e-81b402213205 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.284418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324e9ef-e009-4d1c-8b0e-81b402213205 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.286374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324e9ef-e009-4d1c-8b0e-81b402213205 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.295754 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324e9ef-e009-4d1c-8b0e-81b402213205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:26.302783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-2324e9ef-e009-4d1c-8b0e-81b402213205 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:26.303981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:26.304900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:26.305683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 255/508] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:26.306331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324e9ef-e009-4d1c-8b0e-81b402213205 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.375250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151731f1-3009-44e1-b56a-6905c6d70561 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.403547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324e9ef-e009-4d1c-8b0e-81b402213205 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.411287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb4494-ac8f-41c0-bc4d-96726ed999dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.411957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb4494-ac8f-41c0-bc4d-96726ed999dc None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.412243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb4494-ac8f-41c0-bc4d-96726ed999dc None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.412518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb4494-ac8f-41c0-bc4d-96726ed999dc None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.413379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb4494-ac8f-41c0-bc4d-96726ed999dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:26.413686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb4494-ac8f-41c0-bc4d-96726ed999dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.415554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb4494-ac8f-41c0-bc4d-96726ed999dc None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.435617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 254/509] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Jul 24 19:41:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:26.436285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb4494-ac8f-41c0-bc4d-96726ed999dc None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.441629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701dcc5-4318-485a-a32b-48079031288e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.442301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701dcc5-4318-485a-a32b-48079031288e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.442583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701dcc5-4318-485a-a32b-48079031288e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.442857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701dcc5-4318-485a-a32b-48079031288e None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.443586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701dcc5-4318-485a-a32b-48079031288e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:26.443900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701dcc5-4318-485a-a32b-48079031288e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.445297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701dcc5-4318-485a-a32b-48079031288e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.463914 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 256/510] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:26.463914 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701dcc5-4318-485a-a32b-48079031288e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.510332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:26.543297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:26.554022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb4494-ac8f-41c0-bc4d-96726ed999dc None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.565586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701dcc5-4318-485a-a32b-48079031288e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.587273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a655ba-34bc-4840-aed9-c7fcd2b19c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.587936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a655ba-34bc-4840-aed9-c7fcd2b19c3b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.588250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a655ba-34bc-4840-aed9-c7fcd2b19c3b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.588602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a655ba-34bc-4840-aed9-c7fcd2b19c3b None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.589462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a655ba-34bc-4840-aed9-c7fcd2b19c3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:26.589768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a655ba-34bc-4840-aed9-c7fcd2b19c3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.591202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a655ba-34bc-4840-aed9-c7fcd2b19c3b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.598794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ccf75-50f0-46ab-b3f6-69d08c58671e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.601123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ccf75-50f0-46ab-b3f6-69d08c58671e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.601123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ccf75-50f0-46ab-b3f6-69d08c58671e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.601123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ccf75-50f0-46ab-b3f6-69d08c58671e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.609786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 255/511] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Jul 24 19:41:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:26.610618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a655ba-34bc-4840-aed9-c7fcd2b19c3b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.669513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ccf75-50f0-46ab-b3f6-69d08c58671e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.670882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ccf75-50f0-46ab-b3f6-69d08c58671e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.713032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a655ba-34bc-4840-aed9-c7fcd2b19c3b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.716618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:26.752325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e76c43-a135-4a2d-b754-4777da7b777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.753027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e76c43-a135-4a2d-b754-4777da7b777e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.753307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e76c43-a135-4a2d-b754-4777da7b777e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.753581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e76c43-a135-4a2d-b754-4777da7b777e None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.754330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e76c43-a135-4a2d-b754-4777da7b777e None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:26.754627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e76c43-a135-4a2d-b754-4777da7b777e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.756097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e76c43-a135-4a2d-b754-4777da7b777e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.764039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 256/512] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:26.764868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e76c43-a135-4a2d-b754-4777da7b777e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.776511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:26.783142 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 257/513] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:26.783142 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ccf75-50f0-46ab-b3f6-69d08c58671e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:26.864669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e76c43-a135-4a2d-b754-4777da7b777e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.879905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ccf75-50f0-46ab-b3f6-69d08c58671e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:26.897293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25568e75-0647-4270-a30d-3b47ccee525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.897931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25568e75-0647-4270-a30d-3b47ccee525b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.898215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25568e75-0647-4270-a30d-3b47ccee525b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.898485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25568e75-0647-4270-a30d-3b47ccee525b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.913649 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1592efb-676b-45aa-a15e-297e78e32335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:26.914308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592efb-676b-45aa-a15e-297e78e32335 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:26.914591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592efb-676b-45aa-a15e-297e78e32335 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:26.914867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592efb-676b-45aa-a15e-297e78e32335 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:26.915563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1592efb-676b-45aa-a15e-297e78e32335 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.917853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1592efb-676b-45aa-a15e-297e78e32335 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.965794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25568e75-0647-4270-a30d-3b47ccee525b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:26.967222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25568e75-0647-4270-a30d-3b47ccee525b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:26.977912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:27.048674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 258/514] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:41:26 2022] POST /identity/v3/users => generated 247 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:27.048674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1592efb-676b-45aa-a15e-297e78e32335 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.050774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:27.072577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 257/515] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:27.074070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25568e75-0647-4270-a30d-3b47ccee525b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.181155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1592efb-676b-45aa-a15e-297e78e32335 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:27.189738 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25568e75-0647-4270-a30d-3b47ccee525b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:27.201930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:27.227118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb998d-f6b5-4f69-b530-bd33a38abe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:27.227918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb998d-f6b5-4f69-b530-bd33a38abe34 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:27.228201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb998d-f6b5-4f69-b530-bd33a38abe34 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:27.229370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb998d-f6b5-4f69-b530-bd33a38abe34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:27.231101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b7d41-5555-4ec2-911e-5212ac079828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:27.231833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b7d41-5555-4ec2-911e-5212ac079828 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:27.232124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b7d41-5555-4ec2-911e-5212ac079828 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:27.232401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b7d41-5555-4ec2-911e-5212ac079828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:27.240037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb998d-f6b5-4f69-b530-bd33a38abe34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:27.241561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb998d-f6b5-4f69-b530-bd33a38abe34 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-65fb998d-f6b5-4f69-b530-bd33a38abe34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:27.263315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:27.264198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:27.265088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:27.265088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:27.265088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:27.265088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:27.265088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 258/516] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:27.265355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb998d-f6b5-4f69-b530-bd33a38abe34 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.303960 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b7d41-5555-4ec2-911e-5212ac079828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:27.305445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b7d41-5555-4ec2-911e-5212ac079828 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:27.308152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:27.366382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb998d-f6b5-4f69-b530-bd33a38abe34 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:27.404412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ceddb-98a0-43b8-8179-9ad57b29cc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:27.405242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ceddb-98a0-43b8-8179-9ad57b29cc7d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:27.405530 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ceddb-98a0-43b8-8179-9ad57b29cc7d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:27.405808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ceddb-98a0-43b8-8179-9ad57b29cc7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:27.417638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 259/517] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:27.418677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b7d41-5555-4ec2-911e-5212ac079828 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.426690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ceddb-98a0-43b8-8179-9ad57b29cc7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:27.428066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ceddb-98a0-43b8-8179-9ad57b29cc7d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-749ceddb-98a0-43b8-8179-9ad57b29cc7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:27.446771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:27.448193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:27.449088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:27.449088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:27.449088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:27.449088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:27.449088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 259/518] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:27.449387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ceddb-98a0-43b8-8179-9ad57b29cc7d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.468313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:27.535020 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b7d41-5555-4ec2-911e-5212ac079828 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:27.538523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:27.563742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ceddb-98a0-43b8-8179-9ad57b29cc7d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:27.577544 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7c505-7268-4851-b0b1-8e416cf795f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:27.578691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7c505-7268-4851-b0b1-8e416cf795f3 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:27.579481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7c505-7268-4851-b0b1-8e416cf795f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:27.580169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7c505-7268-4851-b0b1-8e416cf795f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:27.592859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7c505-7268-4851-b0b1-8e416cf795f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:27.594319 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7c505-7268-4851-b0b1-8e416cf795f3 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:27.600255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332aa8ef-891b-48e2-baaf-32d9df3c7f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:27.601421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332aa8ef-891b-48e2-baaf-32d9df3c7f85 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:27.601717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332aa8ef-891b-48e2-baaf-32d9df3c7f85 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:27.602003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332aa8ef-891b-48e2-baaf-32d9df3c7f85 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:27.603147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332aa8ef-891b-48e2-baaf-32d9df3c7f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:27.603471 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332aa8ef-891b-48e2-baaf-32d9df3c7f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:27.604919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332aa8ef-891b-48e2-baaf-32d9df3c7f85 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-ded7c505-7268-4851-b0b1-8e416cf795f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:27.611881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:27.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:27.620223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:27.620223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:27.620223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:27.620223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:27.620223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 260/519] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:27.620223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7c505-7268-4851-b0b1-8e416cf795f3 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.620549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 260/520] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:27.620549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332aa8ef-891b-48e2-baaf-32d9df3c7f85 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.706534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:27.724534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7c505-7268-4851-b0b1-8e416cf795f3 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:27.734940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332aa8ef-891b-48e2-baaf-32d9df3c7f85 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:27.759783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d9613-dec7-4129-9d44-ca4cf913f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:27.761207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9613-dec7-4129-9d44-ca4cf913f183 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:27.761501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9613-dec7-4129-9d44-ca4cf913f183 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:27.761778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9613-dec7-4129-9d44-ca4cf913f183 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:27.771849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d9613-dec7-4129-9d44-ca4cf913f183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:27.773343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d9613-dec7-4129-9d44-ca4cf913f183 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:27.778254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a617-fd17-4534-a57a-102e7d9fe36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:27.779266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a617-fd17-4534-a57a-102e7d9fe36a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:27.779733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a617-fd17-4534-a57a-102e7d9fe36a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:27.780619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a617-fd17-4534-a57a-102e7d9fe36a None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:27.781564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a617-fd17-4534-a57a-102e7d9fe36a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:27.782412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a617-fd17-4534-a57a-102e7d9fe36a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:27.784352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a617-fd17-4534-a57a-102e7d9fe36a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-727d9613-dec7-4129-9d44-ca4cf913f183 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:27.791141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:27.792274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:27.793263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:27.793263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:27.793263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:27.793263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:27.793263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 261/521] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:27.793923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d9613-dec7-4129-9d44-ca4cf913f183 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.808169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 261/522] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:27.808169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a617-fd17-4534-a57a-102e7d9fe36a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.811476 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:27.915054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d9613-dec7-4129-9d44-ca4cf913f183 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:27.920296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a617-fd17-4534-a57a-102e7d9fe36a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:27.954434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85216-ae22-4af1-a8c8-61cee4eedcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:27.954591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fff99c-7c57-4230-bd55-c8bcda081a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:27.955183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fff99c-7c57-4230-bd55-c8bcda081a7f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:27.955284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85216-ae22-4af1-a8c8-61cee4eedcde None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:27.955472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fff99c-7c57-4230-bd55-c8bcda081a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:27.955582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85216-ae22-4af1-a8c8-61cee4eedcde None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:27.955750 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fff99c-7c57-4230-bd55-c8bcda081a7f None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:27.955859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85216-ae22-4af1-a8c8-61cee4eedcde None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:27.956474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fff99c-7c57-4230-bd55-c8bcda081a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:27.956814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fff99c-7c57-4230-bd55-c8bcda081a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:27.958116 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:27.958218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fff99c-7c57-4230-bd55-c8bcda081a7f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:27.964229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc85216-ae22-4af1-a8c8-61cee4eedcde None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:27.965681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc85216-ae22-4af1-a8c8-61cee4eedcde None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:27.974543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 262/523] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:27.974543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fff99c-7c57-4230-bd55-c8bcda081a7f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-edc85216-ae22-4af1-a8c8-61cee4eedcde None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:27.983980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:27.985069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:27.986403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 262/524] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:27.990523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85216-ae22-4af1-a8c8-61cee4eedcde None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.057977 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:28.092484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fff99c-7c57-4230-bd55-c8bcda081a7f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.102148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85216-ae22-4af1-a8c8-61cee4eedcde None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.133199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727c723-c13c-4880-b499-d31d10ff623f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:28.134187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727c723-c13c-4880-b499-d31d10ff623f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:28.134655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727c723-c13c-4880-b499-d31d10ff623f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:28.135112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727c723-c13c-4880-b499-d31d10ff623f None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:28.135981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727c723-c13c-4880-b499-d31d10ff623f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:28.136461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727c723-c13c-4880-b499-d31d10ff623f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:28.137995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727c723-c13c-4880-b499-d31d10ff623f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:28.138109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc2192-96f9-4470-b609-a88f57939593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:28.138833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc2192-96f9-4470-b609-a88f57939593 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:28.139123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc2192-96f9-4470-b609-a88f57939593 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:28.139402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc2192-96f9-4470-b609-a88f57939593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:28.146323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc2192-96f9-4470-b609-a88f57939593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:28.147808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc2192-96f9-4470-b609-a88f57939593 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:28.151892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 263/525] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:28.152960 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727c723-c13c-4880-b499-d31d10ff623f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-f9dc2192-96f9-4470-b609-a88f57939593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:28.168685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:28.169567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 263/526] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:28.170466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc2192-96f9-4470-b609-a88f57939593 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.233508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:28.263485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727c723-c13c-4880-b499-d31d10ff623f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.281219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc2192-96f9-4470-b609-a88f57939593 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.290725 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:28.300093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb7e6c-d79d-4e78-81bd-7dafe80f7219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:28.300913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb7e6c-d79d-4e78-81bd-7dafe80f7219 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:28.301198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb7e6c-d79d-4e78-81bd-7dafe80f7219 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:28.301471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb7e6c-d79d-4e78-81bd-7dafe80f7219 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:28.309286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb7e6c-d79d-4e78-81bd-7dafe80f7219 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:28.310702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb7e6c-d79d-4e78-81bd-7dafe80f7219 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:28.316577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4b073-1f9f-4067-9614-42f927a1852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:28.317266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4b073-1f9f-4067-9614-42f927a1852b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:28.317550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4b073-1f9f-4067-9614-42f927a1852b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:28.317827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4b073-1f9f-4067-9614-42f927a1852b None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:28.318658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4b073-1f9f-4067-9614-42f927a1852b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:28.318968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4b073-1f9f-4067-9614-42f927a1852b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:28.320397 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4b073-1f9f-4067-9614-42f927a1852b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-93fb7e6c-d79d-4e78-81bd-7dafe80f7219 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:28.327250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:28.328212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:28.329258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:28.329258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:28.329258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:28.329258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:28.329258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:28.329258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:28.329258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:28.329258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:28.331283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 264/527] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:28.332255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb7e6c-d79d-4e78-81bd-7dafe80f7219 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.339149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 264/528] 10.222.0.17 () {56 vars in 1139 bytes} [Sun Jul 24 19:41:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:28.340064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4b073-1f9f-4067-9614-42f927a1852b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.450312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4b073-1f9f-4067-9614-42f927a1852b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.451031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb7e6c-d79d-4e78-81bd-7dafe80f7219 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.464401 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:28.486092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fc92f-a7f3-4c8e-a856-48294b8039be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:28.493694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fc92f-a7f3-4c8e-a856-48294b8039be None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:28.493694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fc92f-a7f3-4c8e-a856-48294b8039be None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:28.493889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890eaa7-2938-49d9-9021-192c94bc8a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:28.493989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fc92f-a7f3-4c8e-a856-48294b8039be None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:28.493989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fc92f-a7f3-4c8e-a856-48294b8039be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:28.494145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890eaa7-2938-49d9-9021-192c94bc8a96 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:28.494225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fc92f-a7f3-4c8e-a856-48294b8039be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:28.494341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890eaa7-2938-49d9-9021-192c94bc8a96 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:28.494341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890eaa7-2938-49d9-9021-192c94bc8a96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:28.494499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fc92f-a7f3-4c8e-a856-48294b8039be None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:28.495491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890eaa7-2938-49d9-9021-192c94bc8a96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:28.496970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890eaa7-2938-49d9-9021-192c94bc8a96 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:28.513815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 265/529] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:28.513815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fc92f-a7f3-4c8e-a856-48294b8039be None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-7890eaa7-2938-49d9-9021-192c94bc8a96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:28.514025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:28.514806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 265/530] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:28.515642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890eaa7-2938-49d9-9021-192c94bc8a96 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.566554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:28.622149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890eaa7-2938-49d9-9021-192c94bc8a96 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.623349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fc92f-a7f3-4c8e-a856-48294b8039be None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.658549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9866efd-6948-44bf-8378-ebddce119d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:28.659233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9866efd-6948-44bf-8378-ebddce119d77 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:28.659521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9866efd-6948-44bf-8378-ebddce119d77 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:28.659802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9866efd-6948-44bf-8378-ebddce119d77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:28.661434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d544827-55b6-462b-8e3a-a9d49387d518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:28.662094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d544827-55b6-462b-8e3a-a9d49387d518 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:28.662376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d544827-55b6-462b-8e3a-a9d49387d518 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:28.662645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d544827-55b6-462b-8e3a-a9d49387d518 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:28.663430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d544827-55b6-462b-8e3a-a9d49387d518 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:28.663736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d544827-55b6-462b-8e3a-a9d49387d518 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:28.665193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d544827-55b6-462b-8e3a-a9d49387d518 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:28.678519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 266/531] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:28.679153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d544827-55b6-462b-8e3a-a9d49387d518 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.712174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:28.738630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9866efd-6948-44bf-8378-ebddce119d77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:28.740279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9866efd-6948-44bf-8378-ebddce119d77 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:28.792275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d544827-55b6-462b-8e3a-a9d49387d518 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.794096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:28.827999 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c2def-2c0c-46bc-9da4-df3107e581ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:28.828576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c2def-2c0c-46bc-9da4-df3107e581ff None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:28.828892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c2def-2c0c-46bc-9da4-df3107e581ff None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:28.829173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c2def-2c0c-46bc-9da4-df3107e581ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:28.830009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c2def-2c0c-46bc-9da4-df3107e581ff None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=fe7a67bd815147afa3f186e95b004469) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:28.830330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c2def-2c0c-46bc-9da4-df3107e581ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:28.831781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c2def-2c0c-46bc-9da4-df3107e581ff None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:28.837264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 267/532] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:28 2022] GET /identity/v3/role_assignments?user.id=fe7a67bd815147afa3f186e95b004469&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:28.838137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c2def-2c0c-46bc-9da4-df3107e581ff None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.862460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 266/533] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:28.863640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9866efd-6948-44bf-8378-ebddce119d77 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:28.964767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:28.973463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c2def-2c0c-46bc-9da4-df3107e581ff None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:28.996502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9866efd-6948-44bf-8378-ebddce119d77 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:29.025509 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d7e14-b947-4b49-8ab2-1bf9b2cefdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:29.027070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7e14-b947-4b49-8ab2-1bf9b2cefdde None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.027537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7e14-b947-4b49-8ab2-1bf9b2cefdde None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.027813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7e14-b947-4b49-8ab2-1bf9b2cefdde None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.028599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7e14-b947-4b49-8ab2-1bf9b2cefdde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:29.036731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7e14-b947-4b49-8ab2-1bf9b2cefdde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:29.036731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7e14-b947-4b49-8ab2-1bf9b2cefdde None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:29.045665 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044045b-bc2c-437b-9708-f816b5e2bd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:29.046451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044045b-bc2c-437b-9708-f816b5e2bd12 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.046859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044045b-bc2c-437b-9708-f816b5e2bd12 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.047235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044045b-bc2c-437b-9708-f816b5e2bd12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.047753 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 268/534] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:29.048274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044045b-bc2c-437b-9708-f816b5e2bd12 None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=123b7f659c6a4ceba18ce268b1335542) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:29.048673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044045b-bc2c-437b-9708-f816b5e2bd12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:29.048775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d7e14-b947-4b49-8ab2-1bf9b2cefdde None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:29.050377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044045b-bc2c-437b-9708-f816b5e2bd12 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:29.057069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 267/535] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:28 2022] GET /identity/v3/role_assignments?user.id=123b7f659c6a4ceba18ce268b1335542&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:29.058112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044045b-bc2c-437b-9708-f816b5e2bd12 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:29.075721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:29.181602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d7e14-b947-4b49-8ab2-1bf9b2cefdde None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:29.219956 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e173d-a770-4a6e-a477-87a1d3939595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:29.220797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e173d-a770-4a6e-a477-87a1d3939595 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.221084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e173d-a770-4a6e-a477-87a1d3939595 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.221361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e173d-a770-4a6e-a477-87a1d3939595 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.233195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e173d-a770-4a6e-a477-87a1d3939595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:29.234679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e173d-a770-4a6e-a477-87a1d3939595 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:29.235638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044045b-bc2c-437b-9708-f816b5e2bd12 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-845e173d-a770-4a6e-a477-87a1d3939595 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:29.253925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:29.254868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:29.255698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:29.255698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:29.255698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:29.255698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:29.255698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 269/536] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:29.256406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448f5e3-40c2-4b83-9a31-42dbfa909b3f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.256686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448f5e3-40c2-4b83-9a31-42dbfa909b3f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.256967 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448f5e3-40c2-4b83-9a31-42dbfa909b3f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.258471 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 270/537] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:29.261437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448f5e3-40c2-4b83-9a31-42dbfa909b3f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:29.273034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:29.281887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10e7bd-b9d2-4bf9-8035-7fce6ea40c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:29.282555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10e7bd-b9d2-4bf9-8035-7fce6ea40c34 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.282843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10e7bd-b9d2-4bf9-8035-7fce6ea40c34 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.290386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10e7bd-b9d2-4bf9-8035-7fce6ea40c34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.290386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10e7bd-b9d2-4bf9-8035-7fce6ea40c34 None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=bca33ef49ba44832a7fb45b2317d211c) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:29.290386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10e7bd-b9d2-4bf9-8035-7fce6ea40c34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:29.290386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10e7bd-b9d2-4bf9-8035-7fce6ea40c34 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:29.292523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 268/538] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:29 2022] GET /identity/v3/role_assignments?user.id=bca33ef49ba44832a7fb45b2317d211c&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:29.293784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a2386-f18d-441d-976b-1855756fb76f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.294209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a2386-f18d-441d-976b-1855756fb76f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.294494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a2386-f18d-441d-976b-1855756fb76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.313288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-881a2386-f18d-441d-976b-1855756fb76f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:29.314791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-881a2386-f18d-441d-976b-1855756fb76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:29.404676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 269/539] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:29.418871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a2386-f18d-441d-976b-1855756fb76f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:29.432773 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:29.457443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448f5e3-40c2-4b83-9a31-42dbfa909b3f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:29.533309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0cfff-72ef-449a-be49-d01c23204dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:29.533986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0cfff-72ef-449a-be49-d01c23204dfc None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.534271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0cfff-72ef-449a-be49-d01c23204dfc None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.534547 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0cfff-72ef-449a-be49-d01c23204dfc None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.534767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a2386-f18d-441d-976b-1855756fb76f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:29.535282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0cfff-72ef-449a-be49-d01c23204dfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:29.535590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0cfff-72ef-449a-be49-d01c23204dfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:29.537249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0cfff-72ef-449a-be49-d01c23204dfc None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:29.553757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 271/540] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:29.559534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0cfff-72ef-449a-be49-d01c23204dfc None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:29.583406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ac74b-7d5d-4657-9487-b89b2e0a31b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:29.584224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ac74b-7d5d-4657-9487-b89b2e0a31b2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.584512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ac74b-7d5d-4657-9487-b89b2e0a31b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.584823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ac74b-7d5d-4657-9487-b89b2e0a31b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.596929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ac74b-7d5d-4657-9487-b89b2e0a31b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-450ac74b-7d5d-4657-9487-b89b2e0a31b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:29.604929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:29.606058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:29.606855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:29.611080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 270/541] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:29.612296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ac74b-7d5d-4657-9487-b89b2e0a31b2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:29.680362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:29.718019 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ac74b-7d5d-4657-9487-b89b2e0a31b2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:29.745906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:29.753145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9579d-daea-430e-bc33-23cfd2e08404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:29.753835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9579d-daea-430e-bc33-23cfd2e08404 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.754128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9579d-daea-430e-bc33-23cfd2e08404 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.754414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9579d-daea-430e-bc33-23cfd2e08404 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.755180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9579d-daea-430e-bc33-23cfd2e08404 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:29.755499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9579d-daea-430e-bc33-23cfd2e08404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:29.756925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9579d-daea-430e-bc33-23cfd2e08404 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:29.770876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 271/542] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:29.772122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9579d-daea-430e-bc33-23cfd2e08404 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:29.816392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0cfff-72ef-449a-be49-d01c23204dfc None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:29.878501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9579d-daea-430e-bc33-23cfd2e08404 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:29.881585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526f2ff-2cc8-4741-8445-73cbc2a27ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:29.882622 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526f2ff-2cc8-4741-8445-73cbc2a27ad2 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.883120 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526f2ff-2cc8-4741-8445-73cbc2a27ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.883585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526f2ff-2cc8-4741-8445-73cbc2a27ad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.897334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526f2ff-2cc8-4741-8445-73cbc2a27ad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:29.898821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526f2ff-2cc8-4741-8445-73cbc2a27ad2 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:29.919152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20df6f-facd-4532-abf3-81877f4784d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:29.919961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20df6f-facd-4532-abf3-81877f4784d1 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:29.920258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20df6f-facd-4532-abf3-81877f4784d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:29.920545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20df6f-facd-4532-abf3-81877f4784d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-6526f2ff-2cc8-4741-8445-73cbc2a27ad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:29.923015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:29.923804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:29.924624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:29.924624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:29.924624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:29.924624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:29.924624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:29.924624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:29.924624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:29.924624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:29.924624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 272/543] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:29.931316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526f2ff-2cc8-4741-8445-73cbc2a27ad2 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:29.932781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20df6f-facd-4532-abf3-81877f4784d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:29.934271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20df6f-facd-4532-abf3-81877f4784d1 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:29.944274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-2c20df6f-facd-4532-abf3-81877f4784d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:29.959651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:29.960486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:29.961348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:29.961348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:29.961348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:29.961348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:29.965699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 272/544] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:29.967226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20df6f-facd-4532-abf3-81877f4784d1 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:30.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20df6f-facd-4532-abf3-81877f4784d1 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:30.127422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ac6a-25fa-4306-86c4-69bc2209a9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:30.128106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ac6a-25fa-4306-86c4-69bc2209a9d2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:30.128398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ac6a-25fa-4306-86c4-69bc2209a9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:30.128700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ac6a-25fa-4306-86c4-69bc2209a9d2 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:30.129439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38ac6a-25fa-4306-86c4-69bc2209a9d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:30.129753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38ac6a-25fa-4306-86c4-69bc2209a9d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:30.131134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38ac6a-25fa-4306-86c4-69bc2209a9d2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:30.158639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 273/545] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:30.163781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ac6a-25fa-4306-86c4-69bc2209a9d2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:30.209436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526f2ff-2cc8-4741-8445-73cbc2a27ad2 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:30.265159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:30.271823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:30.290288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ccf52-f77b-46d7-b047-7fbf6cc45691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:30.290961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ccf52-f77b-46d7-b047-7fbf6cc45691 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:30.292077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ccf52-f77b-46d7-b047-7fbf6cc45691 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:30.292365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ccf52-f77b-46d7-b047-7fbf6cc45691 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:30.293232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ccf52-f77b-46d7-b047-7fbf6cc45691 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:30.293553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ccf52-f77b-46d7-b047-7fbf6cc45691 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:30.295008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ccf52-f77b-46d7-b047-7fbf6cc45691 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:30.313990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 273/546] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:30.319762 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ccf52-f77b-46d7-b047-7fbf6cc45691 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:30.367422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ac6a-25fa-4306-86c4-69bc2209a9d2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:30.442400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ccf52-f77b-46d7-b047-7fbf6cc45691 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:30.461863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87678510-cbdb-4607-a92b-fdb60576fc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:30.462669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87678510-cbdb-4607-a92b-fdb60576fc21 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:30.462997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87678510-cbdb-4607-a92b-fdb60576fc21 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:30.463328 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87678510-cbdb-4607-a92b-fdb60576fc21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:30.482122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87678510-cbdb-4607-a92b-fdb60576fc21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:30.483572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87678510-cbdb-4607-a92b-fdb60576fc21 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:30.487892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b24ca-a82a-4946-89d0-476cccad1dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:30.488548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b24ca-a82a-4946-89d0-476cccad1dc6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:30.488855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b24ca-a82a-4946-89d0-476cccad1dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:30.489131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b24ca-a82a-4946-89d0-476cccad1dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:30.490811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b24ca-a82a-4946-89d0-476cccad1dc6 None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=b9763f9899214df3929f3816f009e857) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:30.491129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b24ca-a82a-4946-89d0-476cccad1dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:30.492547 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b24ca-a82a-4946-89d0-476cccad1dc6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-87678510-cbdb-4607-a92b-fdb60576fc21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:30.507656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:30.508775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:30.509691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 274/547] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:30.515784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 274/548] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:30 2022] GET /identity/v3/role_assignments?user.id=b9763f9899214df3929f3816f009e857&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:30.516536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87678510-cbdb-4607-a92b-fdb60576fc21 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:30.661336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:30.670980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87678510-cbdb-4607-a92b-fdb60576fc21 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:30.715098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18964f45-02e3-43e8-aecc-3db795457f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:30.715788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18964f45-02e3-43e8-aecc-3db795457f13 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:30.716134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18964f45-02e3-43e8-aecc-3db795457f13 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:30.716417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18964f45-02e3-43e8-aecc-3db795457f13 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:30.717298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18964f45-02e3-43e8-aecc-3db795457f13 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:30.717613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18964f45-02e3-43e8-aecc-3db795457f13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:30.719059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18964f45-02e3-43e8-aecc-3db795457f13 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:30.768193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 275/549] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Jul 24 19:41:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:30.772552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b24ca-a82a-4946-89d0-476cccad1dc6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:30.833155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:30.889531 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b24ca-a82a-4946-89d0-476cccad1dc6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:30.936215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71225e9-28f5-4d7c-95d2-899bc2ee1a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:30.936832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71225e9-28f5-4d7c-95d2-899bc2ee1a82 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:30.937123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71225e9-28f5-4d7c-95d2-899bc2ee1a82 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:30.937405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71225e9-28f5-4d7c-95d2-899bc2ee1a82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.023218 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71225e9-28f5-4d7c-95d2-899bc2ee1a82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:31.028727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71225e9-28f5-4d7c-95d2-899bc2ee1a82 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:31.092014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:31.167104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 275/550] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:31.168470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18964f45-02e3-43e8-aecc-3db795457f13 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:31.200702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043168f5-35cf-41c6-b4f1-56e6c7b5ce37 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.200981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043168f5-35cf-41c6-b4f1-56e6c7b5ce37 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.201389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043168f5-35cf-41c6-b4f1-56e6c7b5ce37 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.203527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 276/551] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:41:31.208826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3674cd-1d93-4bf4-bc81-d9484d0bc0d6 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.209122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3674cd-1d93-4bf4-bc81-d9484d0bc0d6 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.209403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3674cd-1d93-4bf4-bc81-d9484d0bc0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.227098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2f3674cd-1d93-4bf4-bc81-d9484d0bc0d6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:31.228322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2f3674cd-1d93-4bf4-bc81-d9484d0bc0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:31.306092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 277/552] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:31.324757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3674cd-1d93-4bf4-bc81-d9484d0bc0d6 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:31.332256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18964f45-02e3-43e8-aecc-3db795457f13 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:31.375349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc12394-b3df-4443-8dd9-5d5a1a261b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:31.376157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc12394-b3df-4443-8dd9-5d5a1a261b62 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.376447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc12394-b3df-4443-8dd9-5d5a1a261b62 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.376751 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc12394-b3df-4443-8dd9-5d5a1a261b62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.391855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc12394-b3df-4443-8dd9-5d5a1a261b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:31.393396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc12394-b3df-4443-8dd9-5d5a1a261b62 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-4cc12394-b3df-4443-8dd9-5d5a1a261b62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:31.431529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:31.432428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:31.433306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:31.433306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:31.433306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:31.433306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:31.433306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 276/553] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:31.453735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc12394-b3df-4443-8dd9-5d5a1a261b62 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:31.460995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:31.468313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:31.498206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3674cd-1d93-4bf4-bc81-d9484d0bc0d6 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:31.566041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d81a8-08c9-4720-b574-886cd9095cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:31.566850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d81a8-08c9-4720-b574-886cd9095cc3 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.567142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d81a8-08c9-4720-b574-886cd9095cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.567425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d81a8-08c9-4720-b574-886cd9095cc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.598227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d81a8-08c9-4720-b574-886cd9095cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-693d81a8-08c9-4720-b574-886cd9095cc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:31.617124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:31.618048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:31.618872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:31.620319 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 278/554] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:31.622777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d81a8-08c9-4720-b574-886cd9095cc3 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:31.643623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc12394-b3df-4443-8dd9-5d5a1a261b62 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:31.687743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d74b1e-59fb-4564-8e7c-40001baba057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:31.688415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d74b1e-59fb-4564-8e7c-40001baba057 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.688742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d74b1e-59fb-4564-8e7c-40001baba057 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.689120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d74b1e-59fb-4564-8e7c-40001baba057 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.689853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d74b1e-59fb-4564-8e7c-40001baba057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:31.690168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d74b1e-59fb-4564-8e7c-40001baba057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:31.691658 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d74b1e-59fb-4564-8e7c-40001baba057 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:31.706338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 277/555] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:31.707674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b26e35-c5a8-47c5-961c-4345666dd13d None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.707969 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b26e35-c5a8-47c5-961c-4345666dd13d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.708253 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b26e35-c5a8-47c5-961c-4345666dd13d None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.712994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 278/556] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:31.713860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b26e35-c5a8-47c5-961c-4345666dd13d None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:31.763590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d81a8-08c9-4720-b574-886cd9095cc3 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:31.802619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938899a-503c-4bf7-b2d2-9f819b92308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:31.803300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938899a-503c-4bf7-b2d2-9f819b92308c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.803681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938899a-503c-4bf7-b2d2-9f819b92308c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.803965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938899a-503c-4bf7-b2d2-9f819b92308c None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.804802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938899a-503c-4bf7-b2d2-9f819b92308c None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:31.805194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938899a-503c-4bf7-b2d2-9f819b92308c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:31.806763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:31.808311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938899a-503c-4bf7-b2d2-9f819b92308c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:31.817430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 279/557] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:31.818907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88fd71-49ec-469b-8a14-44de96ec14c7 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.819234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88fd71-49ec-469b-8a14-44de96ec14c7 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.820716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88fd71-49ec-469b-8a14-44de96ec14c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.826692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:31.837014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1a88fd71-49ec-469b-8a14-44de96ec14c7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:31.838238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1a88fd71-49ec-469b-8a14-44de96ec14c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:31.843046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b26e35-c5a8-47c5-961c-4345666dd13d None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:31.882103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd029d8c-6c74-46f3-9f31-0e25fafe083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:31.883282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd029d8c-6c74-46f3-9f31-0e25fafe083d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.883564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd029d8c-6c74-46f3-9f31-0e25fafe083d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.883894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd029d8c-6c74-46f3-9f31-0e25fafe083d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.890883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd029d8c-6c74-46f3-9f31-0e25fafe083d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:31.892326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd029d8c-6c74-46f3-9f31-0e25fafe083d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-bd029d8c-6c74-46f3-9f31-0e25fafe083d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:31.906990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:31.908346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:31.909248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:31.909248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:31.909248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:31.909248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:31.909248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:31.909248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:31.909248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:31.909248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:31.911540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 279/558] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:31.913033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b640b0d-2e64-49d0-9b46-28462d9b067b None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:31.913517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b640b0d-2e64-49d0-9b46-28462d9b067b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:31.913983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b640b0d-2e64-49d0-9b46-28462d9b067b None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:31.915716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 280/559] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:31.917489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b640b0d-2e64-49d0-9b46-28462d9b067b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:31.924590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 280/560] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:31.925527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88fd71-49ec-469b-8a14-44de96ec14c7 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.035895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b640b0d-2e64-49d0-9b46-28462d9b067b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.039184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88fd71-49ec-469b-8a14-44de96ec14c7 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.069318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:32.075828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e3deb-9483-46dd-b82c-33ac7a6ff865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e3deb-9483-46dd-b82c-33ac7a6ff865 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.076932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e3deb-9483-46dd-b82c-33ac7a6ff865 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.077204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e3deb-9483-46dd-b82c-33ac7a6ff865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.077304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d8b0e-6054-4014-be72-c9c0945ffacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.077986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d8b0e-6054-4014-be72-c9c0945ffacc None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.078278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d8b0e-6054-4014-be72-c9c0945ffacc None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.078552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d8b0e-6054-4014-be72-c9c0945ffacc None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.079335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d8b0e-6054-4014-be72-c9c0945ffacc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:32.079642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d8b0e-6054-4014-be72-c9c0945ffacc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:32.081097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d8b0e-6054-4014-be72-c9c0945ffacc None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:32.082755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:32.090913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e3deb-9483-46dd-b82c-33ac7a6ff865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:32.092414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e3deb-9483-46dd-b82c-33ac7a6ff865 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:32.098546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 281/561] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:32.098546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e828e-1ef0-4992-a31e-eba374db5cff None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.098546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e828e-1ef0-4992-a31e-eba374db5cff None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.098546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e828e-1ef0-4992-a31e-eba374db5cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-d10e3deb-9483-46dd-b82c-33ac7a6ff865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:32.112347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:32.113461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:32.120392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:32.120392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:32.120392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:32.120392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:32.120392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 281/562] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:32.120392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e3deb-9483-46dd-b82c-33ac7a6ff865 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.120920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-0e6e828e-1ef0-4992-a31e-eba374db5cff None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:32.120920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0e6e828e-1ef0-4992-a31e-eba374db5cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:32.196669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 282/563] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:32.196669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e828e-1ef0-4992-a31e-eba374db5cff None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.225195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e3deb-9483-46dd-b82c-33ac7a6ff865 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.261132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be98337-3c53-4c13-a5de-0e4299bf4ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.261931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be98337-3c53-4c13-a5de-0e4299bf4ed7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.262219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be98337-3c53-4c13-a5de-0e4299bf4ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.262495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be98337-3c53-4c13-a5de-0e4299bf4ed7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.272864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be98337-3c53-4c13-a5de-0e4299bf4ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-8be98337-3c53-4c13-a5de-0e4299bf4ed7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:32.281269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:32.282270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:32.283107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:32.287128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 282/564] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:32.288121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be98337-3c53-4c13-a5de-0e4299bf4ed7 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.300994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e828e-1ef0-4992-a31e-eba374db5cff None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.325458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:32.335724 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:32.336927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcca08a-9c3e-4972-8a49-e1bf8730357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.337590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcca08a-9c3e-4972-8a49-e1bf8730357e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.337874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcca08a-9c3e-4972-8a49-e1bf8730357e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.338724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcca08a-9c3e-4972-8a49-e1bf8730357e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.339554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcca08a-9c3e-4972-8a49-e1bf8730357e None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=0849f31902da4613babb10a76c4548f3) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:32.339868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcca08a-9c3e-4972-8a49-e1bf8730357e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:32.341662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcca08a-9c3e-4972-8a49-e1bf8730357e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:32.348694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 283/565] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/role_assignments?user.id=0849f31902da4613babb10a76c4548f3&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:32.348694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcca08a-9c3e-4972-8a49-e1bf8730357e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.429512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be98337-3c53-4c13-a5de-0e4299bf4ed7 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.474113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463db40-57a0-47da-96c3-824f587aa11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.475132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463db40-57a0-47da-96c3-824f587aa11d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.475488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463db40-57a0-47da-96c3-824f587aa11d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.475860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463db40-57a0-47da-96c3-824f587aa11d None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.476793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463db40-57a0-47da-96c3-824f587aa11d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:32.477209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463db40-57a0-47da-96c3-824f587aa11d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:32.479270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463db40-57a0-47da-96c3-824f587aa11d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:32.485976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcca08a-9c3e-4972-8a49-e1bf8730357e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.498493 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 283/566] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:32.499571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463db40-57a0-47da-96c3-824f587aa11d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.529216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34da7c-8617-4987-89b3-b9e0ba54547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.530182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34da7c-8617-4987-89b3-b9e0ba54547d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.530827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34da7c-8617-4987-89b3-b9e0ba54547d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.531198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34da7c-8617-4987-89b3-b9e0ba54547d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.544886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34da7c-8617-4987-89b3-b9e0ba54547d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-4f34da7c-8617-4987-89b3-b9e0ba54547d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:32.560326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:32.561360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:32.562180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:32.564715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 284/567] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:32.565738 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34da7c-8617-4987-89b3-b9e0ba54547d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.619654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463db40-57a0-47da-96c3-824f587aa11d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.621502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:32.631211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:32.657661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1a010-dc5c-4a1c-81da-b86bd6b51332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.658335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1a010-dc5c-4a1c-81da-b86bd6b51332 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.658619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1a010-dc5c-4a1c-81da-b86bd6b51332 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.658946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1a010-dc5c-4a1c-81da-b86bd6b51332 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.659691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1a010-dc5c-4a1c-81da-b86bd6b51332 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:32.659998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1a010-dc5c-4a1c-81da-b86bd6b51332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:32.662028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1a010-dc5c-4a1c-81da-b86bd6b51332 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:32.680730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 284/568] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:32.683128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1a010-dc5c-4a1c-81da-b86bd6b51332 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.704396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34da7c-8617-4987-89b3-b9e0ba54547d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.741612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368cfb0-f327-48be-a004-5ccdfc59d3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.742404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368cfb0-f327-48be-a004-5ccdfc59d3a7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.742823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368cfb0-f327-48be-a004-5ccdfc59d3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.743058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368cfb0-f327-48be-a004-5ccdfc59d3a7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.751689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368cfb0-f327-48be-a004-5ccdfc59d3a7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:32.753230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368cfb0-f327-48be-a004-5ccdfc59d3a7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-8368cfb0-f327-48be-a004-5ccdfc59d3a7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:32.769147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:32.770125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 285/569] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:32.771467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368cfb0-f327-48be-a004-5ccdfc59d3a7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.822991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1a010-dc5c-4a1c-81da-b86bd6b51332 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.865239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f505b28-dc49-426e-831a-f51f88afec7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.865956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f505b28-dc49-426e-831a-f51f88afec7d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.866243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f505b28-dc49-426e-831a-f51f88afec7d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.866518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f505b28-dc49-426e-831a-f51f88afec7d None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.867264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f505b28-dc49-426e-831a-f51f88afec7d None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:32.867586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f505b28-dc49-426e-831a-f51f88afec7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:32.869637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f505b28-dc49-426e-831a-f51f88afec7d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:32.883617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 285/570] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:32.884542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f505b28-dc49-426e-831a-f51f88afec7d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.904988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368cfb0-f327-48be-a004-5ccdfc59d3a7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:32.914568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:32.914904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:32.941082 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948d531-7b6c-4209-86a9-3ebf657f2760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:32.941870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948d531-7b6c-4209-86a9-3ebf657f2760 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:32.942161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948d531-7b6c-4209-86a9-3ebf657f2760 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:32.942447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948d531-7b6c-4209-86a9-3ebf657f2760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:32.953822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948d531-7b6c-4209-86a9-3ebf657f2760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:32.955276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948d531-7b6c-4209-86a9-3ebf657f2760 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a948d531-7b6c-4209-86a9-3ebf657f2760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:32.972414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:32.973524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:32.974410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:32.974410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:32.974410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:32.974410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:32.974410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 286/571] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:32.974686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948d531-7b6c-4209-86a9-3ebf657f2760 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:32.994808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f505b28-dc49-426e-831a-f51f88afec7d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.035316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e57a-1ab5-4362-9fa3-a2143920b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.035992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e57a-1ab5-4362-9fa3-a2143920b4ab None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.036274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e57a-1ab5-4362-9fa3-a2143920b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.036851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e57a-1ab5-4362-9fa3-a2143920b4ab None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.037402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e57a-1ab5-4362-9fa3-a2143920b4ab None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:33.037784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e57a-1ab5-4362-9fa3-a2143920b4ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:33.039130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e57a-1ab5-4362-9fa3-a2143920b4ab None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:33.058535 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 286/572] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:33.062379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230a328-a6dc-4f58-8725-c2db4e0f7884 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.063401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230a328-a6dc-4f58-8725-c2db4e0f7884 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.063685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230a328-a6dc-4f58-8725-c2db4e0f7884 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.065131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 287/573] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:33.069731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230a328-a6dc-4f58-8725-c2db4e0f7884 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.082644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948d531-7b6c-4209-86a9-3ebf657f2760 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.117756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d43f9d-e768-437d-85b5-9110669828ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.118555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d43f9d-e768-437d-85b5-9110669828ba None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.118845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d43f9d-e768-437d-85b5-9110669828ba None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.119660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d43f9d-e768-437d-85b5-9110669828ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.131391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d43f9d-e768-437d-85b5-9110669828ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:33.133065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d43f9d-e768-437d-85b5-9110669828ba None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-82d43f9d-e768-437d-85b5-9110669828ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:33.151080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:33.152150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:33.158634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:33.158634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:33.158634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:33.158634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:33.158634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 287/574] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:33.158634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d43f9d-e768-437d-85b5-9110669828ba None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.170021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:33.176546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:33.193360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230a328-a6dc-4f58-8725-c2db4e0f7884 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.227688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b92bea-b2ad-4769-9b25-06539483d04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.228337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92bea-b2ad-4769-9b25-06539483d04c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.229180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92bea-b2ad-4769-9b25-06539483d04c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.229455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92bea-b2ad-4769-9b25-06539483d04c None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.230177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b92bea-b2ad-4769-9b25-06539483d04c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:33.230865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b92bea-b2ad-4769-9b25-06539483d04c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:33.232950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b92bea-b2ad-4769-9b25-06539483d04c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:33.245124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 288/575] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:33.246180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd86bb-aba9-43f0-8ff8-82959de22d1a None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.246469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd86bb-aba9-43f0-8ff8-82959de22d1a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.246749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd86bb-aba9-43f0-8ff8-82959de22d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.262961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-23fd86bb-aba9-43f0-8ff8-82959de22d1a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:33.264208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-23fd86bb-aba9-43f0-8ff8-82959de22d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:33.266935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d43f9d-e768-437d-85b5-9110669828ba None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.301583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c51144-9fe9-43fe-ba5b-ed6a8ed7b77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.302381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c51144-9fe9-43fe-ba5b-ed6a8ed7b77c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.302665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c51144-9fe9-43fe-ba5b-ed6a8ed7b77c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.302947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c51144-9fe9-43fe-ba5b-ed6a8ed7b77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.313941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c51144-9fe9-43fe-ba5b-ed6a8ed7b77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:33.315393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c51144-9fe9-43fe-ba5b-ed6a8ed7b77c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-d3c51144-9fe9-43fe-ba5b-ed6a8ed7b77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:33.332732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:33.333771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:33.334859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:33.334859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:33.334859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:33.334859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:33.338338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 288/576] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:33.339175 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c51144-9fe9-43fe-ba5b-ed6a8ed7b77c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.352663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 289/577] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:33.352663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd86bb-aba9-43f0-8ff8-82959de22d1a None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.419194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:33.436056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:33.457235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c51144-9fe9-43fe-ba5b-ed6a8ed7b77c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd86bb-aba9-43f0-8ff8-82959de22d1a None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.496247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d738c9b-1326-498d-ad3e-d448206a7176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.496940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d738c9b-1326-498d-ad3e-d448206a7176 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.497224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d738c9b-1326-498d-ad3e-d448206a7176 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.497497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d738c9b-1326-498d-ad3e-d448206a7176 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.498351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d738c9b-1326-498d-ad3e-d448206a7176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:33.498656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d738c9b-1326-498d-ad3e-d448206a7176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:33.500025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d738c9b-1326-498d-ad3e-d448206a7176 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:33.503763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61045a43-4b91-48b7-8389-af3727773d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.504574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61045a43-4b91-48b7-8389-af3727773d12 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.505288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61045a43-4b91-48b7-8389-af3727773d12 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.505926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61045a43-4b91-48b7-8389-af3727773d12 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.516560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 289/578] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:33.516560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d738c9b-1326-498d-ad3e-d448206a7176 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.518157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61045a43-4b91-48b7-8389-af3727773d12 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:33.519599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61045a43-4b91-48b7-8389-af3727773d12 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-61045a43-4b91-48b7-8389-af3727773d12 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:33.534777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:33.535730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:33.536728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 290/579] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:33.537316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61045a43-4b91-48b7-8389-af3727773d12 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.627198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d738c9b-1326-498d-ad3e-d448206a7176 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.640859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61045a43-4b91-48b7-8389-af3727773d12 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.652196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:33.664036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4ef2-90c9-4f0c-87a7-24460c036a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.665258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4ef2-90c9-4f0c-87a7-24460c036a8a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.669488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4ef2-90c9-4f0c-87a7-24460c036a8a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.669488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4ef2-90c9-4f0c-87a7-24460c036a8a None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.669488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb4ef2-90c9-4f0c-87a7-24460c036a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:33.669488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb4ef2-90c9-4f0c-87a7-24460c036a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:33.669985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb4ef2-90c9-4f0c-87a7-24460c036a8a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:33.676292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afadd69-72ff-45a2-b984-aae0819c2132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.677596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afadd69-72ff-45a2-b984-aae0819c2132 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.677596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afadd69-72ff-45a2-b984-aae0819c2132 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.677787 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afadd69-72ff-45a2-b984-aae0819c2132 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.679939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:33.682887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 290/580] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:33.683514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4ef2-90c9-4f0c-87a7-24460c036a8a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.696353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afadd69-72ff-45a2-b984-aae0819c2132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-3afadd69-72ff-45a2-b984-aae0819c2132 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:33.704889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:33.705826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:33.706641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 291/581] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:33.707406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afadd69-72ff-45a2-b984-aae0819c2132 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.791486 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4ef2-90c9-4f0c-87a7-24460c036a8a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.811110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afadd69-72ff-45a2-b984-aae0819c2132 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.826871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2a47f-8164-43ef-b193-947a09a8ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.827679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2a47f-8164-43ef-b193-947a09a8ea77 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.827963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2a47f-8164-43ef-b193-947a09a8ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.828357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2a47f-8164-43ef-b193-947a09a8ea77 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.836743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2a47f-8164-43ef-b193-947a09a8ea77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:33.838456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2a47f-8164-43ef-b193-947a09a8ea77 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:33.845733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbbe1d-ca10-4549-9d9b-43b46b15789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:33.846409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbbe1d-ca10-4549-9d9b-43b46b15789a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:33.846687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbbe1d-ca10-4549-9d9b-43b46b15789a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:33.846959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbbe1d-ca10-4549-9d9b-43b46b15789a None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:33.847791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbbe1d-ca10-4549-9d9b-43b46b15789a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:33.848098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbbe1d-ca10-4549-9d9b-43b46b15789a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:33.849549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbbe1d-ca10-4549-9d9b-43b46b15789a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-cba2a47f-8164-43ef-b193-947a09a8ea77 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:33.853739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:33.854639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 24 19:41:33.855565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:33.860099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 291/582] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:33.860545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2a47f-8164-43ef-b193-947a09a8ea77 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.869778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 292/583] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:33.870647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbbe1d-ca10-4549-9d9b-43b46b15789a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:33.914854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:33.923233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:33.970619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2a47f-8164-43ef-b193-947a09a8ea77 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:33.973117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbbe1d-ca10-4549-9d9b-43b46b15789a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.007545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3dc9a-bfbd-4408-b137-b6c32237c312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.008340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3dc9a-bfbd-4408-b137-b6c32237c312 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.008623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3dc9a-bfbd-4408-b137-b6c32237c312 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.008924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3dc9a-bfbd-4408-b137-b6c32237c312 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.011025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd4d90-441b-496c-9d1f-731196ba7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.011697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd4d90-441b-496c-9d1f-731196ba7563 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.011981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd4d90-441b-496c-9d1f-731196ba7563 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.012257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd4d90-441b-496c-9d1f-731196ba7563 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.013041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd4d90-441b-496c-9d1f-731196ba7563 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:34.013353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd4d90-441b-496c-9d1f-731196ba7563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.014719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd4d90-441b-496c-9d1f-731196ba7563 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.015878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3dc9a-bfbd-4408-b137-b6c32237c312 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.017359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3dc9a-bfbd-4408-b137-b6c32237c312 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.023271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 293/584] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:34.023910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd4d90-441b-496c-9d1f-731196ba7563 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-53a3dc9a-bfbd-4408-b137-b6c32237c312 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:34.035460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:34.036342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:34.037286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:34.037286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:34.037286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:34.037286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:34.037286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:34.037286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:34.037286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:34.037286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:34.037286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 292/585] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:34.037773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3dc9a-bfbd-4408-b137-b6c32237c312 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.147819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd4d90-441b-496c-9d1f-731196ba7563 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.149004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3dc9a-bfbd-4408-b137-b6c32237c312 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.152436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:34.163288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:34.185897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da8e49-d4e8-4cbe-a0af-4ba8a1a5a95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.186541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da8e49-d4e8-4cbe-a0af-4ba8a1a5a95d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.186796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da8e49-d4e8-4cbe-a0af-4ba8a1a5a95d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.187121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da8e49-d4e8-4cbe-a0af-4ba8a1a5a95d None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.187665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edac19-afb9-48d7-bfee-27b4c923d8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.187967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da8e49-d4e8-4cbe-a0af-4ba8a1a5a95d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:34.188331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da8e49-d4e8-4cbe-a0af-4ba8a1a5a95d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.188464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edac19-afb9-48d7-bfee-27b4c923d8e6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.188849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edac19-afb9-48d7-bfee-27b4c923d8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.189101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edac19-afb9-48d7-bfee-27b4c923d8e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.189832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da8e49-d4e8-4cbe-a0af-4ba8a1a5a95d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.200215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edac19-afb9-48d7-bfee-27b4c923d8e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.201708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edac19-afb9-48d7-bfee-27b4c923d8e6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.214362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 294/586] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:34.214362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da8e49-d4e8-4cbe-a0af-4ba8a1a5a95d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-29edac19-afb9-48d7-bfee-27b4c923d8e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:34.220778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:34.221652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:34.222518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:34.222518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:34.222518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:34.222518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:34.226484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 293/587] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:34.227322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edac19-afb9-48d7-bfee-27b4c923d8e6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.321397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da8e49-d4e8-4cbe-a0af-4ba8a1a5a95d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.342359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edac19-afb9-48d7-bfee-27b4c923d8e6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.364800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8e0d4-2ecb-4b12-b3f6-4089bc7af0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.365624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8e0d4-2ecb-4b12-b3f6-4089bc7af0d2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.365986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8e0d4-2ecb-4b12-b3f6-4089bc7af0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.366268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8e0d4-2ecb-4b12-b3f6-4089bc7af0d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.377785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8e0d4-2ecb-4b12-b3f6-4089bc7af0d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.379224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8e0d4-2ecb-4b12-b3f6-4089bc7af0d2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.379488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2d4c6-e981-4b3e-9b8d-18c9ddbef55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.380145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2d4c6-e981-4b3e-9b8d-18c9ddbef55e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.380431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2d4c6-e981-4b3e-9b8d-18c9ddbef55e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.380740 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2d4c6-e981-4b3e-9b8d-18c9ddbef55e None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.381523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2d4c6-e981-4b3e-9b8d-18c9ddbef55e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:34.381836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2d4c6-e981-4b3e-9b8d-18c9ddbef55e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.383246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2d4c6-e981-4b3e-9b8d-18c9ddbef55e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.395371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 294/588] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-0cc8e0d4-2ecb-4b12-b3f6-4089bc7af0d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:34.399211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:34.400426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:34.406488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:34.406488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:34.406488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:34.406488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:34.406851 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2d4c6-e981-4b3e-9b8d-18c9ddbef55e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.406938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 295/589] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:34.406938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8e0d4-2ecb-4b12-b3f6-4089bc7af0d2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.424534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:34.431424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:34.505448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8e0d4-2ecb-4b12-b3f6-4089bc7af0d2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.506053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2d4c6-e981-4b3e-9b8d-18c9ddbef55e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.538577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c623e-a371-4c4e-8376-8c961ee10f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.539361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c623e-a371-4c4e-8376-8c961ee10f3b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.539623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd4d3d-6569-4b10-bbff-77c0948f8bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.539735 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c623e-a371-4c4e-8376-8c961ee10f3b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.539931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c623e-a371-4c4e-8376-8c961ee10f3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.540283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd4d3d-6569-4b10-bbff-77c0948f8bf9 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.540571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd4d3d-6569-4b10-bbff-77c0948f8bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.540877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd4d3d-6569-4b10-bbff-77c0948f8bf9 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.542400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd4d3d-6569-4b10-bbff-77c0948f8bf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:34.542714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd4d3d-6569-4b10-bbff-77c0948f8bf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.544092 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd4d3d-6569-4b10-bbff-77c0948f8bf9 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.551142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c623e-a371-4c4e-8376-8c961ee10f3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.552562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c623e-a371-4c4e-8376-8c961ee10f3b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.555147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 296/590] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:34.556371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7d307-4aae-4310-896c-ed626c21b81b None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.556717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7d307-4aae-4310-896c-ed626c21b81b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.557001 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7d307-4aae-4310-896c-ed626c21b81b None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.561121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 297/591] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:34.561121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7d307-4aae-4310-896c-ed626c21b81b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-ce1c623e-a371-4c4e-8376-8c961ee10f3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:34.572696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:34.573758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:34.574820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:34.574820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:34.574820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:34.574820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:34.576934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 295/592] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:34.577778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c623e-a371-4c4e-8376-8c961ee10f3b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.643024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:34.649401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:34.665964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7d307-4aae-4310-896c-ed626c21b81b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.676792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c623e-a371-4c4e-8376-8c961ee10f3b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.700299 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b25926-d693-4da0-9372-cf6479247173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.701226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b25926-d693-4da0-9372-cf6479247173 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.701509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b25926-d693-4da0-9372-cf6479247173 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.701793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b25926-d693-4da0-9372-cf6479247173 None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/fe7a67bd815147afa3f186e95b004469/roles/65e4ab8e291442e3858317b0aa6c2d8a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.710248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d41669-557a-4890-8b53-90d1779633e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.710909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d41669-557a-4890-8b53-90d1779633e9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.711197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d41669-557a-4890-8b53-90d1779633e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.711472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d41669-557a-4890-8b53-90d1779633e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.712191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d41669-557a-4890-8b53-90d1779633e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:34.712501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d41669-557a-4890-8b53-90d1779633e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.714455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d41669-557a-4890-8b53-90d1779633e9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.716474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b25926-d693-4da0-9372-cf6479247173 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=fe7a67bd815147afa3f186e95b004469, role_id=65e4ab8e291442e3858317b0aa6c2d8a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.717945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b25926-d693-4da0-9372-cf6479247173 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.726010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 296/593] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:34.726822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d41669-557a-4890-8b53-90d1779633e9 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.822799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 298/594] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:41:34 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/fe7a67bd815147afa3f186e95b004469/roles/65e4ab8e291442e3858317b0aa6c2d8a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:34.822799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2469-5f49-4ce9-9b65-8ada33804c2c None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.822799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2469-5f49-4ce9-9b65-8ada33804c2c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.822799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2469-5f49-4ce9-9b65-8ada33804c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.825233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d41669-557a-4890-8b53-90d1779633e9 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:34.845505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d85c2469-5f49-4ce9-9b65-8ada33804c2c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:34.846745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d85c2469-5f49-4ce9-9b65-8ada33804c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:34.859147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:34.869969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e084ecf-d0d2-482a-bf0b-a5487c25898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:34.871135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e084ecf-d0d2-482a-bf0b-a5487c25898d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:34.871434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e084ecf-d0d2-482a-bf0b-a5487c25898d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:34.871804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e084ecf-d0d2-482a-bf0b-a5487c25898d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:34.882762 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e084ecf-d0d2-482a-bf0b-a5487c25898d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:34.884374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e084ecf-d0d2-482a-bf0b-a5487c25898d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-0e084ecf-d0d2-482a-bf0b-a5487c25898d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:34.900127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:34.901741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:34.902677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:34.902677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:34.902677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:34.902677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:34.902677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:34.902677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:34.902677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:34.902677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:34.904514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 297/595] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:34.905538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e084ecf-d0d2-482a-bf0b-a5487c25898d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:34.925302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 299/596] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:34.926883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c2469-5f49-4ce9-9b65-8ada33804c2c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.011370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:35.029654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e084ecf-d0d2-482a-bf0b-a5487c25898d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.050472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c2469-5f49-4ce9-9b65-8ada33804c2c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.065590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d063-ba00-47e5-a493-fcd566f21367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.066643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d063-ba00-47e5-a493-fcd566f21367 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.066938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d063-ba00-47e5-a493-fcd566f21367 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.067219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d063-ba00-47e5-a493-fcd566f21367 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.067955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d063-ba00-47e5-a493-fcd566f21367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:35.072909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d063-ba00-47e5-a493-fcd566f21367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:35.072909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d063-ba00-47e5-a493-fcd566f21367 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:35.084143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 298/597] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:35.088680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d063-ba00-47e5-a493-fcd566f21367 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.091328 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c8955-af67-45a4-b241-bd6bb2ae679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.092331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c8955-af67-45a4-b241-bd6bb2ae679c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.092829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c8955-af67-45a4-b241-bd6bb2ae679c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.093113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c8955-af67-45a4-b241-bd6bb2ae679c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.103216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c8955-af67-45a4-b241-bd6bb2ae679c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:35.105005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c8955-af67-45a4-b241-bd6bb2ae679c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-2f8c8955-af67-45a4-b241-bd6bb2ae679c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:35.124036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:35.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 300/598] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:35.132322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c8955-af67-45a4-b241-bd6bb2ae679c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.150825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:35.207517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d063-ba00-47e5-a493-fcd566f21367 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.233214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c8955-af67-45a4-b241-bd6bb2ae679c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.242213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c53e9-abbd-453f-8116-c599c42d1f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.242880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c53e9-abbd-453f-8116-c599c42d1f6e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.243166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c53e9-abbd-453f-8116-c599c42d1f6e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.243444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c53e9-abbd-453f-8116-c599c42d1f6e None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.244805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c53e9-abbd-453f-8116-c599c42d1f6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:35.245122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c53e9-abbd-453f-8116-c599c42d1f6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:35.246548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c53e9-abbd-453f-8116-c599c42d1f6e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:35.262397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:35.263705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 299/599] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:35.264470 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c53e9-abbd-453f-8116-c599c42d1f6e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.278775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97656964-df05-450a-abad-2cbd42cae6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.279576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97656964-df05-450a-abad-2cbd42cae6e8 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.279866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97656964-df05-450a-abad-2cbd42cae6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.280141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97656964-df05-450a-abad-2cbd42cae6e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.292286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97656964-df05-450a-abad-2cbd42cae6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-97656964-df05-450a-abad-2cbd42cae6e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:35.300292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:35.301390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:35.302425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:35.311194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 301/600] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:35.312101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97656964-df05-450a-abad-2cbd42cae6e8 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.375168 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c53e9-abbd-453f-8116-c599c42d1f6e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.384618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:35.413811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4e9b3-30b4-40ba-82d4-5046abaf4c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.414701 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4e9b3-30b4-40ba-82d4-5046abaf4c92 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.415058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4e9b3-30b4-40ba-82d4-5046abaf4c92 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.415408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4e9b3-30b4-40ba-82d4-5046abaf4c92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.415530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97656964-df05-450a-abad-2cbd42cae6e8 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.423766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4e9b3-30b4-40ba-82d4-5046abaf4c92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:35.425346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4e9b3-30b4-40ba-82d4-5046abaf4c92 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-82a4e9b3-30b4-40ba-82d4-5046abaf4c92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:35.441059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:35.442148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:35.443229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:35.443229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:35.443229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:35.443229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:35.443229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:35.443229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:35.443229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:35.443229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:35.444137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 300/601] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:35.444137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4e9b3-30b4-40ba-82d4-5046abaf4c92 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.460633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05538-721f-4e21-b9d7-491fdb367410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.461330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05538-721f-4e21-b9d7-491fdb367410 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.461611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05538-721f-4e21-b9d7-491fdb367410 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.461887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05538-721f-4e21-b9d7-491fdb367410 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.462788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c05538-721f-4e21-b9d7-491fdb367410 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:35.463155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c05538-721f-4e21-b9d7-491fdb367410 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:35.464568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c05538-721f-4e21-b9d7-491fdb367410 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:35.482590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 302/602] 10.222.0.17 () {56 vars in 1139 bytes} [Sun Jul 24 19:41:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:35.483429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05538-721f-4e21-b9d7-491fdb367410 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.529057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:35.556208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4e9b3-30b4-40ba-82d4-5046abaf4c92 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.590924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05538-721f-4e21-b9d7-491fdb367410 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.592031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73650494-4708-49e3-a899-a5f316abd9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.592958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73650494-4708-49e3-a899-a5f316abd9d4 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.593237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73650494-4708-49e3-a899-a5f316abd9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.593560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73650494-4708-49e3-a899-a5f316abd9d4 None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/123b7f659c6a4ceba18ce268b1335542/roles/65e4ab8e291442e3858317b0aa6c2d8a` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.608880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73650494-4708-49e3-a899-a5f316abd9d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=123b7f659c6a4ceba18ce268b1335542, role_id=65e4ab8e291442e3858317b0aa6c2d8a)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:35.610309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73650494-4708-49e3-a899-a5f316abd9d4 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:35.627849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7f5db-da01-4a0c-8fbc-4dc7408b2289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.628457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7f5db-da01-4a0c-8fbc-4dc7408b2289 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.628774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7f5db-da01-4a0c-8fbc-4dc7408b2289 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.629055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7f5db-da01-4a0c-8fbc-4dc7408b2289 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.629799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7f5db-da01-4a0c-8fbc-4dc7408b2289 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:35.630108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7f5db-da01-4a0c-8fbc-4dc7408b2289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:35.631457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7f5db-da01-4a0c-8fbc-4dc7408b2289 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:35.639543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 303/603] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:35.640324 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7f5db-da01-4a0c-8fbc-4dc7408b2289 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.692933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 301/604] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:41:35 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/123b7f659c6a4ceba18ce268b1335542/roles/65e4ab8e291442e3858317b0aa6c2d8a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:35.694557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73650494-4708-49e3-a899-a5f316abd9d4 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.725235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:35.768150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7f5db-da01-4a0c-8fbc-4dc7408b2289 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.774908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:35.843231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa834fc3-3c86-44ca-9355-761b3f116279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.844070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa834fc3-3c86-44ca-9355-761b3f116279 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.844452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa834fc3-3c86-44ca-9355-761b3f116279 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.844847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa834fc3-3c86-44ca-9355-761b3f116279 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.845756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa834fc3-3c86-44ca-9355-761b3f116279 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:35.848005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa834fc3-3c86-44ca-9355-761b3f116279 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:35.849619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa834fc3-3c86-44ca-9355-761b3f116279 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:35.889676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 304/605] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:35.891941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa834fc3-3c86-44ca-9355-761b3f116279 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:35.902606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73650494-4708-49e3-a899-a5f316abd9d4 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:35.957614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ea26c-6bae-441a-809d-5bf291aa3ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:35.958810 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ea26c-6bae-441a-809d-5bf291aa3ba0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:35.958810 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ea26c-6bae-441a-809d-5bf291aa3ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:35.959030 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ea26c-6bae-441a-809d-5bf291aa3ba0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:35.979748 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ea26c-6bae-441a-809d-5bf291aa3ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:35.984468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ea26c-6bae-441a-809d-5bf291aa3ba0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:36.013091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa834fc3-3c86-44ca-9355-761b3f116279 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-013ea26c-6bae-441a-809d-5bf291aa3ba0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:36.031265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:36.032207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:36.033146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:36.033146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:36.033146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:36.033146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:36.042741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 302/606] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:36.044661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ea26c-6bae-441a-809d-5bf291aa3ba0 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:36.063385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe5931-2e5c-4843-93bf-69a2aec2b19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:36.064335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe5931-2e5c-4843-93bf-69a2aec2b19d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:36.064742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe5931-2e5c-4843-93bf-69a2aec2b19d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:36.065137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe5931-2e5c-4843-93bf-69a2aec2b19d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:36.076450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe5931-2e5c-4843-93bf-69a2aec2b19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:36.078069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe5931-2e5c-4843-93bf-69a2aec2b19d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-11fe5931-2e5c-4843-93bf-69a2aec2b19d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:36.112732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:36.113840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:36.114949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:36.114949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:36.114949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:36.114949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:36.121425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 305/607] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:36.122383 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe5931-2e5c-4843-93bf-69a2aec2b19d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:36.192627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:36.192987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:36.259212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe5931-2e5c-4843-93bf-69a2aec2b19d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:36.278592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ea26c-6bae-441a-809d-5bf291aa3ba0 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:36.309171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee4756-b619-4720-abf2-9a2f8baf7014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:36.310848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4756-b619-4720-abf2-9a2f8baf7014 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:36.311338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4756-b619-4720-abf2-9a2f8baf7014 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:36.311992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4756-b619-4720-abf2-9a2f8baf7014 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:36.313407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee4756-b619-4720-abf2-9a2f8baf7014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:36.314262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee4756-b619-4720-abf2-9a2f8baf7014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:36.316502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee4756-b619-4720-abf2-9a2f8baf7014 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:36.323213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a860ec-69d4-4dca-a80f-6a1f5b22fa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:36.324236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a860ec-69d4-4dca-a80f-6a1f5b22fa87 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:36.324673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a860ec-69d4-4dca-a80f-6a1f5b22fa87 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:36.325067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a860ec-69d4-4dca-a80f-6a1f5b22fa87 None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/bca33ef49ba44832a7fb45b2317d211c/roles/65e4ab8e291442e3858317b0aa6c2d8a` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:36.337071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 306/608] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:36.338663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee4756-b619-4720-abf2-9a2f8baf7014 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:36.347123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a860ec-69d4-4dca-a80f-6a1f5b22fa87 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=bca33ef49ba44832a7fb45b2317d211c, role_id=65e4ab8e291442e3858317b0aa6c2d8a)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:36.348900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a860ec-69d4-4dca-a80f-6a1f5b22fa87 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:36.430145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:36.449115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 303/609] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:41:36 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/bca33ef49ba44832a7fb45b2317d211c/roles/65e4ab8e291442e3858317b0aa6c2d8a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:36.450891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a860ec-69d4-4dca-a80f-6a1f5b22fa87 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:36.475641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee4756-b619-4720-abf2-9a2f8baf7014 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:36.557800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe14d6-f9f3-4140-807d-6fbc09376c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:36.560157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe14d6-f9f3-4140-807d-6fbc09376c2a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:36.560437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe14d6-f9f3-4140-807d-6fbc09376c2a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:36.560748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe14d6-f9f3-4140-807d-6fbc09376c2a None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:36.561491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe14d6-f9f3-4140-807d-6fbc09376c2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:36.561805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe14d6-f9f3-4140-807d-6fbc09376c2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:36.563256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe14d6-f9f3-4140-807d-6fbc09376c2a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:36.586514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 307/610] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:36.589664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe14d6-f9f3-4140-807d-6fbc09376c2a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:36.675129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a860ec-69d4-4dca-a80f-6a1f5b22fa87 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:36.703776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:36.733068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902f05c-eb1f-4d56-9f2b-1658eca04c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:36.733916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902f05c-eb1f-4d56-9f2b-1658eca04c47 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:36.734210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902f05c-eb1f-4d56-9f2b-1658eca04c47 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:36.734483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902f05c-eb1f-4d56-9f2b-1658eca04c47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:36.747456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902f05c-eb1f-4d56-9f2b-1658eca04c47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:36.751953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902f05c-eb1f-4d56-9f2b-1658eca04c47 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:36.776329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe14d6-f9f3-4140-807d-6fbc09376c2a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-c902f05c-eb1f-4d56-9f2b-1658eca04c47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:36.781682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:36.782534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:36.783406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:36.783406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:36.783406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:36.783406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:36.783406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:36.783406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:36.783406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:36.783406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:36.785960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 304/611] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:36.791320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902f05c-eb1f-4d56-9f2b-1658eca04c47 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:36.820337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e8c14-4daf-446a-8e37-5c8133d229a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:36.821173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e8c14-4daf-446a-8e37-5c8133d229a2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:36.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e8c14-4daf-446a-8e37-5c8133d229a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:36.821730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e8c14-4daf-446a-8e37-5c8133d229a2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:36.832318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:36.837837 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e8c14-4daf-446a-8e37-5c8133d229a2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:36.839479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e8c14-4daf-446a-8e37-5c8133d229a2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-8b9e8c14-4daf-446a-8e37-5c8133d229a2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:36.873331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:36.874368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:36.875774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:36.881150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 308/612] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:36.895084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e8c14-4daf-446a-8e37-5c8133d229a2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:37.010296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e8c14-4daf-446a-8e37-5c8133d229a2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:37.011550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902f05c-eb1f-4d56-9f2b-1658eca04c47 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:37.051571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb7452-9c1d-41ef-b1ca-205cb9950d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:37.051777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e04eb9-896f-4800-a232-d92bba480b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:37.052231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb7452-9c1d-41ef-b1ca-205cb9950d3b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.052359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e04eb9-896f-4800-a232-d92bba480b4c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.052532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e04eb9-896f-4800-a232-d92bba480b4c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.052632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb7452-9c1d-41ef-b1ca-205cb9950d3b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.052810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e04eb9-896f-4800-a232-d92bba480b4c None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.052902 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb7452-9c1d-41ef-b1ca-205cb9950d3b None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.053592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb7452-9c1d-41ef-b1ca-205cb9950d3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:37.053696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e04eb9-896f-4800-a232-d92bba480b4c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:37.053901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb7452-9c1d-41ef-b1ca-205cb9950d3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:37.053999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e04eb9-896f-4800-a232-d92bba480b4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:37.055321 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e04eb9-896f-4800-a232-d92bba480b4c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:37.055441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb7452-9c1d-41ef-b1ca-205cb9950d3b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:37.071004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 305/613] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:37.078369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0ecfd-50f2-4b06-9ce1-f0e53a9befbf None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.078706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0ecfd-50f2-4b06-9ce1-f0e53a9befbf None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.078987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0ecfd-50f2-4b06-9ce1-f0e53a9befbf None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.080411 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 306/614] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:37.090796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0ecfd-50f2-4b06-9ce1-f0e53a9befbf None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:37.090917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 309/615] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Jul 24 19:41:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:37.090917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dabded-e01f-47b6-8e89-d9186b43a5ea None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.090917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dabded-e01f-47b6-8e89-d9186b43a5ea None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.090917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dabded-e01f-47b6-8e89-d9186b43a5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.103851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e6dabded-e01f-47b6-8e89-d9186b43a5ea None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:37.105206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e6dabded-e01f-47b6-8e89-d9186b43a5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:37.131304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:37.166857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:37.196666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0ecfd-50f2-4b06-9ce1-f0e53a9befbf None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:37.196853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 310/616] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:37.196853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dabded-e01f-47b6-8e89-d9186b43a5ea None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:37.247307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce6a6a-2573-4e82-b50b-50ac80ecb8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:37.248337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce6a6a-2573-4e82-b50b-50ac80ecb8b7 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.248752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce6a6a-2573-4e82-b50b-50ac80ecb8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.249099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce6a6a-2573-4e82-b50b-50ac80ecb8b7 None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/b9763f9899214df3929f3816f009e857/roles/65e4ab8e291442e3858317b0aa6c2d8a` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.268377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce6a6a-2573-4e82-b50b-50ac80ecb8b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=b9763f9899214df3929f3816f009e857, role_id=65e4ab8e291442e3858317b0aa6c2d8a)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:37.270038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce6a6a-2573-4e82-b50b-50ac80ecb8b7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:37.323143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dabded-e01f-47b6-8e89-d9186b43a5ea None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:37.353088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 307/617] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:41:37 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/b9763f9899214df3929f3816f009e857/roles/65e4ab8e291442e3858317b0aa6c2d8a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:37.355129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce6a6a-2573-4e82-b50b-50ac80ecb8b7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:37.376972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5077de-ad47-4b1c-b1f3-c9abfa1a8611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:37.377932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5077de-ad47-4b1c-b1f3-c9abfa1a8611 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.378226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5077de-ad47-4b1c-b1f3-c9abfa1a8611 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.378555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5077de-ad47-4b1c-b1f3-c9abfa1a8611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.402227 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5077de-ad47-4b1c-b1f3-c9abfa1a8611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:37.409721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5077de-ad47-4b1c-b1f3-c9abfa1a8611 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-eb5077de-ad47-4b1c-b1f3-c9abfa1a8611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:37.453204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:37.454159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:37.455021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:37.455021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:37.455021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:37.455021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:37.460676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 311/618] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:37.469261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5077de-ad47-4b1c-b1f3-c9abfa1a8611 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:37.479479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:37.492394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce6a6a-2573-4e82-b50b-50ac80ecb8b7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:37.537105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:37.538027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a5ff6-7c90-48c0-b324-4e9a4e3719aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:37.539076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a5ff6-7c90-48c0-b324-4e9a4e3719aa None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.539574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a5ff6-7c90-48c0-b324-4e9a4e3719aa None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.540043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a5ff6-7c90-48c0-b324-4e9a4e3719aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.552546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a5ff6-7c90-48c0-b324-4e9a4e3719aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-3e4a5ff6-7c90-48c0-b324-4e9a4e3719aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:37.560788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:37.561694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:37.562511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:37.565211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 308/619] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:37.572695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a5ff6-7c90-48c0-b324-4e9a4e3719aa None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:37.616615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5077de-ad47-4b1c-b1f3-c9abfa1a8611 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:37.652203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37957a8f-682b-43f9-af52-eb8c97026d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:37.653169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37957a8f-682b-43f9-af52-eb8c97026d90 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.653599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37957a8f-682b-43f9-af52-eb8c97026d90 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.654016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37957a8f-682b-43f9-af52-eb8c97026d90 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.654900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37957a8f-682b-43f9-af52-eb8c97026d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:37.655360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37957a8f-682b-43f9-af52-eb8c97026d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:37.656886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37957a8f-682b-43f9-af52-eb8c97026d90 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:37.670013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 312/620] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:37.676313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37957a8f-682b-43f9-af52-eb8c97026d90 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:37.785520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:37.793327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37957a8f-682b-43f9-af52-eb8c97026d90 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:37.829813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a5ff6-7c90-48c0-b324-4e9a4e3719aa None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:37.831700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561f99-2efe-47c8-a1f3-eb6d9762db8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:37.832713 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561f99-2efe-47c8-a1f3-eb6d9762db8a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.833347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561f99-2efe-47c8-a1f3-eb6d9762db8a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.833767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561f99-2efe-47c8-a1f3-eb6d9762db8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.843604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561f99-2efe-47c8-a1f3-eb6d9762db8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:37.845365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561f99-2efe-47c8-a1f3-eb6d9762db8a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-8e561f99-2efe-47c8-a1f3-eb6d9762db8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:37.863256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:37.864395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:37.865502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:37.865502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:37.865502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:37.865502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:37.865502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:37.865502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:37.865502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:37.865502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:37.867585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 313/621] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:37.873163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561f99-2efe-47c8-a1f3-eb6d9762db8a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:37.924592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891f00-1b69-47ad-86ca-b6a0ace15da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:37.924855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891f00-1b69-47ad-86ca-b6a0ace15da6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.925008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891f00-1b69-47ad-86ca-b6a0ace15da6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.925338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891f00-1b69-47ad-86ca-b6a0ace15da6 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.933357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69891f00-1b69-47ad-86ca-b6a0ace15da6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:37.933693 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69891f00-1b69-47ad-86ca-b6a0ace15da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:37.935086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69891f00-1b69-47ad-86ca-b6a0ace15da6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:37.948197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 309/622] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:37.949599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd19df-b6e7-4e61-8ec8-e7c78db71687 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:37.949894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd19df-b6e7-4e61-8ec8-e7c78db71687 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:37.950228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd19df-b6e7-4e61-8ec8-e7c78db71687 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:37.955089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 310/623] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:37.956249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd19df-b6e7-4e61-8ec8-e7c78db71687 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:37.976624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:37.991947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561f99-2efe-47c8-a1f3-eb6d9762db8a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:38.033506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd563d5-f766-463c-8c35-b305d80982f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:38.034589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd563d5-f766-463c-8c35-b305d80982f4 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.035077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd563d5-f766-463c-8c35-b305d80982f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.035605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd563d5-f766-463c-8c35-b305d80982f4 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.036626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd563d5-f766-463c-8c35-b305d80982f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:38.037211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd563d5-f766-463c-8c35-b305d80982f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:38.038852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd563d5-f766-463c-8c35-b305d80982f4 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:38.051993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 314/624] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:38.053607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cf0fb-ca0e-4de7-95f1-28ac2159deb7 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.054100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cf0fb-ca0e-4de7-95f1-28ac2159deb7 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.054596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cf0fb-ca0e-4de7-95f1-28ac2159deb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.071216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b00cf0fb-ca0e-4de7-95f1-28ac2159deb7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:38.074695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b00cf0fb-ca0e-4de7-95f1-28ac2159deb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:38.078678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd19df-b6e7-4e61-8ec8-e7c78db71687 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:38.085827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:38.118466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac030ce-0e24-4ae2-93f6-cd699586d35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:38.120243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac030ce-0e24-4ae2-93f6-cd699586d35a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.120817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac030ce-0e24-4ae2-93f6-cd699586d35a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.121293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac030ce-0e24-4ae2-93f6-cd699586d35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.133057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac030ce-0e24-4ae2-93f6-cd699586d35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:38.135391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac030ce-0e24-4ae2-93f6-cd699586d35a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-dac030ce-0e24-4ae2-93f6-cd699586d35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:38.155505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:38.156466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:38.157979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:38.157979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:38.157979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:38.157979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:38.158206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 315/625] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:38.158206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cf0fb-ca0e-4de7-95f1-28ac2159deb7 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:38.164433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 311/626] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:38.165964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac030ce-0e24-4ae2-93f6-cd699586d35a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:38.235416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:38.282602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cf0fb-ca0e-4de7-95f1-28ac2159deb7 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:38.284890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac030ce-0e24-4ae2-93f6-cd699586d35a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:38.320520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135109be-036a-4109-b915-b1cd01bef61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:38.322833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135109be-036a-4109-b915-b1cd01bef61c None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.323563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135109be-036a-4109-b915-b1cd01bef61c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.324043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135109be-036a-4109-b915-b1cd01bef61c None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/0849f31902da4613babb10a76c4548f3/roles/65e4ab8e291442e3858317b0aa6c2d8a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.327206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0426e-9e5d-47e4-a809-aead4220fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:38.327867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0426e-9e5d-47e4-a809-aead4220fa4d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.328153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0426e-9e5d-47e4-a809-aead4220fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.328434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0426e-9e5d-47e4-a809-aead4220fa4d None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.330021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0426e-9e5d-47e4-a809-aead4220fa4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:38.330334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0426e-9e5d-47e4-a809-aead4220fa4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:38.331704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0426e-9e5d-47e4-a809-aead4220fa4d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:38.342346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135109be-036a-4109-b915-b1cd01bef61c None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=0849f31902da4613babb10a76c4548f3, role_id=65e4ab8e291442e3858317b0aa6c2d8a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:38.344825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135109be-036a-4109-b915-b1cd01bef61c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:38.350340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 312/627] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:38.350340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0426e-9e5d-47e4-a809-aead4220fa4d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:38.354373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:38.432005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 316/628] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:41:38 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/0849f31902da4613babb10a76c4548f3/roles/65e4ab8e291442e3858317b0aa6c2d8a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:38.433344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135109be-036a-4109-b915-b1cd01bef61c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:38.464951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0426e-9e5d-47e4-a809-aead4220fa4d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:38.496634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:38.531230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ea596-4c38-46da-a52e-ad539025d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:38.532103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ea596-4c38-46da-a52e-ad539025d479 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.532471 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ea596-4c38-46da-a52e-ad539025d479 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.532873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ea596-4c38-46da-a52e-ad539025d479 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.562729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ea596-4c38-46da-a52e-ad539025d479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-fd4ea596-4c38-46da-a52e-ad539025d479 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:38.575731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:38.576743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:38.577665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:38.578887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 313/629] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:38.586088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ea596-4c38-46da-a52e-ad539025d479 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:38.629484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135109be-036a-4109-b915-b1cd01bef61c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:38.681189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d78406-0b6c-48fc-819e-cd94ac95d696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:38.682279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78406-0b6c-48fc-819e-cd94ac95d696 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.682670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78406-0b6c-48fc-819e-cd94ac95d696 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.683097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78406-0b6c-48fc-819e-cd94ac95d696 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.692191 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d78406-0b6c-48fc-819e-cd94ac95d696 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:38.693834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d78406-0b6c-48fc-819e-cd94ac95d696 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-85d78406-0b6c-48fc-819e-cd94ac95d696 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:38.711589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:38.713408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 24 19:41:38.714635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:38.720676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 317/630] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:38.723694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d78406-0b6c-48fc-819e-cd94ac95d696 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:38.737368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ea596-4c38-46da-a52e-ad539025d479 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:38.784161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bac8b3-2c67-4043-96e4-e66f6d44f39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:38.784867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bac8b3-2c67-4043-96e4-e66f6d44f39a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.785156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bac8b3-2c67-4043-96e4-e66f6d44f39a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.785433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bac8b3-2c67-4043-96e4-e66f6d44f39a None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.786200 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bac8b3-2c67-4043-96e4-e66f6d44f39a None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:38.786543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bac8b3-2c67-4043-96e4-e66f6d44f39a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:38.788552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bac8b3-2c67-4043-96e4-e66f6d44f39a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:38.792965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:38.800333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 314/631] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:38.806169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bac8b3-2c67-4043-96e4-e66f6d44f39a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:38.876732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d78406-0b6c-48fc-819e-cd94ac95d696 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:38.880863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:38.915305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4606a-17fd-4757-89f7-3e2eee1296aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:38.915998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4606a-17fd-4757-89f7-3e2eee1296aa None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.916285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4606a-17fd-4757-89f7-3e2eee1296aa None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.916841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4606a-17fd-4757-89f7-3e2eee1296aa None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.917677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4606a-17fd-4757-89f7-3e2eee1296aa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:38.917987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4606a-17fd-4757-89f7-3e2eee1296aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:38.919402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4606a-17fd-4757-89f7-3e2eee1296aa None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:38.937743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 318/632] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Jul 24 19:41:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:38.938943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b13924-93d0-4388-ab88-ae06ed158aa9 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.939232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b13924-93d0-4388-ab88-ae06ed158aa9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.939508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b13924-93d0-4388-ab88-ae06ed158aa9 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:38.941016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 319/633] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:38.942425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bac8b3-2c67-4043-96e4-e66f6d44f39a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:38.945509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b13924-93d0-4388-ab88-ae06ed158aa9 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:38.990780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb03d5-dd95-4515-ae5e-8b74a1db6fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:38.992355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb03d5-dd95-4515-ae5e-8b74a1db6fa7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:38.992889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb03d5-dd95-4515-ae5e-8b74a1db6fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:38.993390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb03d5-dd95-4515-ae5e-8b74a1db6fa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.008796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb03d5-dd95-4515-ae5e-8b74a1db6fa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:39.011491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb03d5-dd95-4515-ae5e-8b74a1db6fa7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-c3cb03d5-dd95-4515-ae5e-8b74a1db6fa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:39.034715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:39.035653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:39.036537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:39.036537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:39.036537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:39.036537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:39.038119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 315/634] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:39.043068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a49dd-7d14-4ebe-baca-66d53f85e2c5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.043068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a49dd-7d14-4ebe-baca-66d53f85e2c5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.043068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a49dd-7d14-4ebe-baca-66d53f85e2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.063272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b13924-93d0-4388-ab88-ae06ed158aa9 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:39.066440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-fc0a49dd-7d14-4ebe-baca-66d53f85e2c5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:39.067883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fc0a49dd-7d14-4ebe-baca-66d53f85e2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:39.079138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:39.101352 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13d07-50a4-4438-89c7-cd284e37c61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:39.102011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13d07-50a4-4438-89c7-cd284e37c61c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.102345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13d07-50a4-4438-89c7-cd284e37c61c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.102731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13d07-50a4-4438-89c7-cd284e37c61c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.156750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 316/635] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:39.157919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a49dd-7d14-4ebe-baca-66d53f85e2c5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:39.189374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:39.202583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c13d07-50a4-4438-89c7-cd284e37c61c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:39.204709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c13d07-50a4-4438-89c7-cd284e37c61c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:39.287671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a49dd-7d14-4ebe-baca-66d53f85e2c5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:39.322260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 320/636] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:39.323186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13d07-50a4-4438-89c7-cd284e37c61c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:39.336115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21482f-d436-44c1-a822-71bc28c6e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:39.337407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21482f-d436-44c1-a822-71bc28c6e626 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.337695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21482f-d436-44c1-a822-71bc28c6e626 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.337980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21482f-d436-44c1-a822-71bc28c6e626 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.338704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21482f-d436-44c1-a822-71bc28c6e626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:39.339016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21482f-d436-44c1-a822-71bc28c6e626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:39.341294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21482f-d436-44c1-a822-71bc28c6e626 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:39.344572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:39.358760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 317/637] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:39.358760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21482f-d436-44c1-a822-71bc28c6e626 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:39.443737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13d07-50a4-4438-89c7-cd284e37c61c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:39.476065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:39.480911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d51d42-3537-443e-9abd-a3abfa7dd5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:39.481718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d51d42-3537-443e-9abd-a3abfa7dd5e1 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.482004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d51d42-3537-443e-9abd-a3abfa7dd5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.482281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d51d42-3537-443e-9abd-a3abfa7dd5e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.492374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21482f-d436-44c1-a822-71bc28c6e626 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:39.493510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d51d42-3537-443e-9abd-a3abfa7dd5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-c0d51d42-3537-443e-9abd-a3abfa7dd5e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:39.502205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:39.503186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:39.504023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:39.508212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 321/638] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:39.509084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d51d42-3537-443e-9abd-a3abfa7dd5e1 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:39.540347 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a47f93-f34c-4b0a-a852-8ec1bc113bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:39.542503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a47f93-f34c-4b0a-a852-8ec1bc113bee None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.542990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a47f93-f34c-4b0a-a852-8ec1bc113bee None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.543846 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a47f93-f34c-4b0a-a852-8ec1bc113bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.558121 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a47f93-f34c-4b0a-a852-8ec1bc113bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:39.560386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a47f93-f34c-4b0a-a852-8ec1bc113bee None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-d8a47f93-f34c-4b0a-a852-8ec1bc113bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:39.583783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:39.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:39.585652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:39.585652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:39.585652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:39.585652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:39.587092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 318/639] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:39.587092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a47f93-f34c-4b0a-a852-8ec1bc113bee None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:39.592669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:39.621983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d51d42-3537-443e-9abd-a3abfa7dd5e1 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:39.658770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c461f2-1900-425c-975e-611db8d92c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:39.659570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c461f2-1900-425c-975e-611db8d92c41 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.659877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c461f2-1900-425c-975e-611db8d92c41 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.660164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c461f2-1900-425c-975e-611db8d92c41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.669008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c461f2-1900-425c-975e-611db8d92c41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:39.670554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c461f2-1900-425c-975e-611db8d92c41 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-d0c461f2-1900-425c-975e-611db8d92c41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:39.684743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:39.685745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 322/640] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Jul 24 19:41:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:39.686823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c461f2-1900-425c-975e-611db8d92c41 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:39.713830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a47f93-f34c-4b0a-a852-8ec1bc113bee None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:39.751524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ffd50-fe84-43a0-9641-238514341dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:39.752194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ffd50-fe84-43a0-9641-238514341dbf None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.752480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ffd50-fe84-43a0-9641-238514341dbf None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.752806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ffd50-fe84-43a0-9641-238514341dbf None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.753554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ffd50-fe84-43a0-9641-238514341dbf None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:39.753857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ffd50-fe84-43a0-9641-238514341dbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:39.755214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ffd50-fe84-43a0-9641-238514341dbf None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:39.763567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 319/641] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:39.764957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa91bc-4299-4fad-ac13-fa27a182528f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.765248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa91bc-4299-4fad-ac13-fa27a182528f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.765530 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa91bc-4299-4fad-ac13-fa27a182528f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.770613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 320/642] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:39.771388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa91bc-4299-4fad-ac13-fa27a182528f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:39.774204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:39.801476 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c461f2-1900-425c-975e-611db8d92c41 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:39.840549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5e3cb-2f11-47a5-9af4-655949559f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:39.841270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5e3cb-2f11-47a5-9af4-655949559f72 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.841555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5e3cb-2f11-47a5-9af4-655949559f72 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.841835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5e3cb-2f11-47a5-9af4-655949559f72 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.842988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5e3cb-2f11-47a5-9af4-655949559f72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:39.843909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5e3cb-2f11-47a5-9af4-655949559f72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:39.845540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5e3cb-2f11-47a5-9af4-655949559f72 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:39.847302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:39.859882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 323/643] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:39.861130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5e3cb-2f11-47a5-9af4-655949559f72 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:39.886959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa91bc-4299-4fad-ac13-fa27a182528f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:39.922979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee3cc3-97ec-458b-9394-a2a37dd711c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:39.923649 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee3cc3-97ec-458b-9394-a2a37dd711c6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.923940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee3cc3-97ec-458b-9394-a2a37dd711c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.924222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee3cc3-97ec-458b-9394-a2a37dd711c6 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.925700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee3cc3-97ec-458b-9394-a2a37dd711c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:39.925700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee3cc3-97ec-458b-9394-a2a37dd711c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:39.926818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee3cc3-97ec-458b-9394-a2a37dd711c6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:39.944719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 321/644] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Jul 24 19:41:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:39.945781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf05e4d-f586-42bb-bc81-89cfcc0884cd None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:39.946072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf05e4d-f586-42bb-bc81-89cfcc0884cd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:39.946352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf05e4d-f586-42bb-bc81-89cfcc0884cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:39.961910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-6cf05e4d-f586-42bb-bc81-89cfcc0884cd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:39.963623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6cf05e4d-f586-42bb-bc81-89cfcc0884cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:39.976721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5e3cb-2f11-47a5-9af4-655949559f72 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.018991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517178df-2f17-4a88-b143-4045c617b586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.019790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517178df-2f17-4a88-b143-4045c617b586 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.020078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517178df-2f17-4a88-b143-4045c617b586 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.020351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517178df-2f17-4a88-b143-4045c617b586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.026076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:40.031439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517178df-2f17-4a88-b143-4045c617b586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:40.032904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517178df-2f17-4a88-b143-4045c617b586 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:40.040756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 322/645] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:40.040756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf05e4d-f586-42bb-bc81-89cfcc0884cd None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-517178df-2f17-4a88-b143-4045c617b586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:40.063459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:40.064374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:40.065474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:40.065474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:40.065474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:40.065474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:40.065474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 324/646] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:40.065755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517178df-2f17-4a88-b143-4045c617b586 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.116206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:40.151054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf05e4d-f586-42bb-bc81-89cfcc0884cd None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.173514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517178df-2f17-4a88-b143-4045c617b586 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.185123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bce3d-d2d4-4525-9ae2-6fb6c9f41ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.185923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bce3d-d2d4-4525-9ae2-6fb6c9f41ea0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.186213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bce3d-d2d4-4525-9ae2-6fb6c9f41ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.186507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bce3d-d2d4-4525-9ae2-6fb6c9f41ea0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.193385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bce3d-d2d4-4525-9ae2-6fb6c9f41ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:40.194935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bce3d-d2d4-4525-9ae2-6fb6c9f41ea0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-e89bce3d-d2d4-4525-9ae2-6fb6c9f41ea0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:40.209015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:40.210105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 323/647] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:40.211011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bce3d-d2d4-4525-9ae2-6fb6c9f41ea0 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.216134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b880-2367-4ebb-9c00-abc017ca4491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.216813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b880-2367-4ebb-9c00-abc017ca4491 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.217100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b880-2367-4ebb-9c00-abc017ca4491 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.217376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b880-2367-4ebb-9c00-abc017ca4491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.264194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:40.294419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b880-2367-4ebb-9c00-abc017ca4491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:40.295834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b880-2367-4ebb-9c00-abc017ca4491 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:40.319305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bce3d-d2d4-4525-9ae2-6fb6c9f41ea0 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.345406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:40.357835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fffca-46bf-4cb9-ba2d-6766613cd179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.358635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fffca-46bf-4cb9-ba2d-6766613cd179 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.359079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fffca-46bf-4cb9-ba2d-6766613cd179 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.359425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fffca-46bf-4cb9-ba2d-6766613cd179 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.371258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fffca-46bf-4cb9-ba2d-6766613cd179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-7e6fffca-46bf-4cb9-ba2d-6766613cd179 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:40.379000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:40.380103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 324/648] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:40.381091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fffca-46bf-4cb9-ba2d-6766613cd179 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.408889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 325/649] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:40.409845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b880-2367-4ebb-9c00-abc017ca4491 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.512283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b880-2367-4ebb-9c00-abc017ca4491 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.512637 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fffca-46bf-4cb9-ba2d-6766613cd179 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.524484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:40.547264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a41a6-5cb8-4ccf-9171-04c544323ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.547998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a41a6-5cb8-4ccf-9171-04c544323ee5 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.548416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a41a6-5cb8-4ccf-9171-04c544323ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.548714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a41a6-5cb8-4ccf-9171-04c544323ee5 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.548896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ba34c-73fd-44f9-b689-87c16c82ac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.549505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a41a6-5cb8-4ccf-9171-04c544323ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:40.549616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ba34c-73fd-44f9-b689-87c16c82ac2f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.549788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ba34c-73fd-44f9-b689-87c16c82ac2f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.549882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a41a6-5cb8-4ccf-9171-04c544323ee5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:40.550065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ba34c-73fd-44f9-b689-87c16c82ac2f None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.550778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ba34c-73fd-44f9-b689-87c16c82ac2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:40.551087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ba34c-73fd-44f9-b689-87c16c82ac2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:40.551240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a41a6-5cb8-4ccf-9171-04c544323ee5 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:40.552443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ba34c-73fd-44f9-b689-87c16c82ac2f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:40.568714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 326/650] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:40.568714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a41a6-5cb8-4ccf-9171-04c544323ee5 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.569000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 325/651] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:40.569000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ba34c-73fd-44f9-b689-87c16c82ac2f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.578840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:40.678521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ba34c-73fd-44f9-b689-87c16c82ac2f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.681009 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a41a6-5cb8-4ccf-9171-04c544323ee5 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.714098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbd797-7c8e-4dc8-b326-97f578e98965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.714888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbd797-7c8e-4dc8-b326-97f578e98965 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.715191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbd797-7c8e-4dc8-b326-97f578e98965 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.715507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbd797-7c8e-4dc8-b326-97f578e98965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.717042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765ec91-f3c9-4d4c-b2a7-956c28af16e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.717834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765ec91-f3c9-4d4c-b2a7-956c28af16e4 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.718174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765ec91-f3c9-4d4c-b2a7-956c28af16e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.718512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765ec91-f3c9-4d4c-b2a7-956c28af16e4 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.719367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765ec91-f3c9-4d4c-b2a7-956c28af16e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:40.719747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765ec91-f3c9-4d4c-b2a7-956c28af16e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:40.721407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765ec91-f3c9-4d4c-b2a7-956c28af16e4 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:40.726998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbd797-7c8e-4dc8-b326-97f578e98965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:40.728995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbd797-7c8e-4dc8-b326-97f578e98965 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:40.734784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 327/652] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:40.734784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765ec91-f3c9-4d4c-b2a7-956c28af16e4 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-cccbd797-7c8e-4dc8-b326-97f578e98965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:40.753008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:40.753938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:40.754814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:40.754814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:40.754814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:40.754814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:40.759370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 326/653] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:40.760331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbd797-7c8e-4dc8-b326-97f578e98965 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.784067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:40.802669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:40.834947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765ec91-f3c9-4d4c-b2a7-956c28af16e4 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.858378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbd797-7c8e-4dc8-b326-97f578e98965 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:40.869408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d959726-0765-4fc5-89a0-56ee4f260878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.870073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d959726-0765-4fc5-89a0-56ee4f260878 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.870355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d959726-0765-4fc5-89a0-56ee4f260878 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.870626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d959726-0765-4fc5-89a0-56ee4f260878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.871453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d959726-0765-4fc5-89a0-56ee4f260878 None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=fe7a67bd815147afa3f186e95b004469) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:40.871759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d959726-0765-4fc5-89a0-56ee4f260878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:40.873212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d959726-0765-4fc5-89a0-56ee4f260878 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:40.880577 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 328/654] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/role_assignments?user.id=fe7a67bd815147afa3f186e95b004469&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:40.881399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d959726-0765-4fc5-89a0-56ee4f260878 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:40.909353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bffe7-726c-4250-bd94-9b0c433a4f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:40.910186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bffe7-726c-4250-bd94-9b0c433a4f0f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:40.910544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bffe7-726c-4250-bd94-9b0c433a4f0f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:40.910825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bffe7-726c-4250-bd94-9b0c433a4f0f None admin] PATH_INFO: `/v3/users/glance` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:40.920038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bffe7-726c-4250-bd94-9b0c433a4f0f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:40.921655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bffe7-726c-4250-bd94-9b0c433a4f0f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-372bffe7-726c-4250-bd94-9b0c433a4f0f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:40.937912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:40.939042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 24 19:41:40.940172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:40.944186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 327/655] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:40.945208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bffe7-726c-4250-bd94-9b0c433a4f0f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.006366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d959726-0765-4fc5-89a0-56ee4f260878 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:41.039667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:41.041451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:41.048933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58931b1e-4f89-49f9-afdb-dce175cdee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:41.053310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58931b1e-4f89-49f9-afdb-dce175cdee10 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.053310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58931b1e-4f89-49f9-afdb-dce175cdee10 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.053310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58931b1e-4f89-49f9-afdb-dce175cdee10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.062555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58931b1e-4f89-49f9-afdb-dce175cdee10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:41.062836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bffe7-726c-4250-bd94-9b0c433a4f0f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:41.064057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58931b1e-4f89-49f9-afdb-dce175cdee10 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-58931b1e-4f89-49f9-afdb-dce175cdee10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:41.083918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:41.084899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:41.085789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:41.085789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:41.085789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:41.085789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:41.085789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 329/656] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:41.086526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b0a68-8290-4792-94f7-4be1b025bf00 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.087384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b0a68-8290-4792-94f7-4be1b025bf00 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.087683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b0a68-8290-4792-94f7-4be1b025bf00 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.092672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 330/657] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:41.092672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b0a68-8290-4792-94f7-4be1b025bf00 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.108435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713cf41-5e04-4c3b-8789-2406e17ccfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:41.109137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713cf41-5e04-4c3b-8789-2406e17ccfa4 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.109425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713cf41-5e04-4c3b-8789-2406e17ccfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.109706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713cf41-5e04-4c3b-8789-2406e17ccfa4 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.110437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713cf41-5e04-4c3b-8789-2406e17ccfa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:41.110750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713cf41-5e04-4c3b-8789-2406e17ccfa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:41.112146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713cf41-5e04-4c3b-8789-2406e17ccfa4 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:41.126662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 328/658] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:41.127711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713cf41-5e04-4c3b-8789-2406e17ccfa4 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.212098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b0a68-8290-4792-94f7-4be1b025bf00 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:41.229302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713cf41-5e04-4c3b-8789-2406e17ccfa4 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:41.250216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee11ab4-6eaa-4a05-ac22-ee3c723289b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:41.251349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee11ab4-6eaa-4a05-ac22-ee3c723289b9 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.251936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee11ab4-6eaa-4a05-ac22-ee3c723289b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.252407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee11ab4-6eaa-4a05-ac22-ee3c723289b9 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.253486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee11ab4-6eaa-4a05-ac22-ee3c723289b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:41.254080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee11ab4-6eaa-4a05-ac22-ee3c723289b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:41.255814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee11ab4-6eaa-4a05-ac22-ee3c723289b9 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:41.262069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9453c-ea82-401b-806c-a1ead96dcc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:41.262739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9453c-ea82-401b-806c-a1ead96dcc10 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.263029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9453c-ea82-401b-806c-a1ead96dcc10 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.263309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9453c-ea82-401b-806c-a1ead96dcc10 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.264040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9453c-ea82-401b-806c-a1ead96dcc10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:41.264353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9453c-ea82-401b-806c-a1ead96dcc10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:41.265748 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9453c-ea82-401b-806c-a1ead96dcc10 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:41.281262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 329/659] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:41.281839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a5029-b325-448d-99ea-3ad4bd6ac756 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.282137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a5029-b325-448d-99ea-3ad4bd6ac756 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.282470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 331/660] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Jul 24 19:41:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:41.283562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee11ab4-6eaa-4a05-ac22-ee3c723289b9 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.284318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a5029-b325-448d-99ea-3ad4bd6ac756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.292499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:41.305280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-817a5029-b325-448d-99ea-3ad4bd6ac756 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:41.308970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-817a5029-b325-448d-99ea-3ad4bd6ac756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:41.338099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:41.383301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 330/661] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:41.383933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a5029-b325-448d-99ea-3ad4bd6ac756 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.399887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee11ab4-6eaa-4a05-ac22-ee3c723289b9 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:41.448184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969d56e-8814-419e-9ccf-dc2ad7a92be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:41.449441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969d56e-8814-419e-9ccf-dc2ad7a92be1 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.450183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969d56e-8814-419e-9ccf-dc2ad7a92be1 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.450634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969d56e-8814-419e-9ccf-dc2ad7a92be1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.459395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969d56e-8814-419e-9ccf-dc2ad7a92be1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:41.460817 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969d56e-8814-419e-9ccf-dc2ad7a92be1 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-8969d56e-8814-419e-9ccf-dc2ad7a92be1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:41.475075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:41.475989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:41.476920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:41.476920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:41.476920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:41.476920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:41.476920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:41.476920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:41.476920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:41.476920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:41.476920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 332/662] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:41.477390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969d56e-8814-419e-9ccf-dc2ad7a92be1 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.506949 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a5029-b325-448d-99ea-3ad4bd6ac756 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:41.538722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:41.544501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81350308-9997-4b91-ab3f-b1f3c87cddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:41.545174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350308-9997-4b91-ab3f-b1f3c87cddc5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.545418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350308-9997-4b91-ab3f-b1f3c87cddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.552409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350308-9997-4b91-ab3f-b1f3c87cddc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.594302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969d56e-8814-419e-9ccf-dc2ad7a92be1 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:41.596678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:41.622173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81350308-9997-4b91-ab3f-b1f3c87cddc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:41.623783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81350308-9997-4b91-ab3f-b1f3c87cddc5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:41.630170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70de636-b103-444c-9c23-475296bae631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:41.630970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70de636-b103-444c-9c23-475296bae631 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.631257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70de636-b103-444c-9c23-475296bae631 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.631537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70de636-b103-444c-9c23-475296bae631 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.641167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70de636-b103-444c-9c23-475296bae631 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:41.642624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70de636-b103-444c-9c23-475296bae631 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-c70de636-b103-444c-9c23-475296bae631 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:41.657829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:41.658880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 24 19:41:41.659805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:41.663694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 333/663] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:41.664568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70de636-b103-444c-9c23-475296bae631 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.733643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 331/664] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:41.734336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81350308-9997-4b91-ab3f-b1f3c87cddc5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.766938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:41.780827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70de636-b103-444c-9c23-475296bae631 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:41.818307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29332c7-14c4-4fc1-ae6b-1e194deaf4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:41.819166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29332c7-14c4-4fc1-ae6b-1e194deaf4da None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.819336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29332c7-14c4-4fc1-ae6b-1e194deaf4da None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.819609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29332c7-14c4-4fc1-ae6b-1e194deaf4da None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.830847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29332c7-14c4-4fc1-ae6b-1e194deaf4da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:41.838594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81350308-9997-4b91-ab3f-b1f3c87cddc5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-f29332c7-14c4-4fc1-ae6b-1e194deaf4da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:41.839083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:41.839991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:41.840897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:41.845366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 334/665] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:41.846069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29332c7-14c4-4fc1-ae6b-1e194deaf4da None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.876702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:41.881058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082e91a-03ee-4c8f-bf5c-e78b858c054d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:41.881632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082e91a-03ee-4c8f-bf5c-e78b858c054d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:41.881920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082e91a-03ee-4c8f-bf5c-e78b858c054d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:41.882195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082e91a-03ee-4c8f-bf5c-e78b858c054d None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:41.883025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082e91a-03ee-4c8f-bf5c-e78b858c054d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:41.883333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082e91a-03ee-4c8f-bf5c-e78b858c054d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:41.885344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082e91a-03ee-4c8f-bf5c-e78b858c054d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:41.904016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 332/666] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:41.904016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082e91a-03ee-4c8f-bf5c-e78b858c054d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:41.964572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29332c7-14c4-4fc1-ae6b-1e194deaf4da None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.000089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbde8e8-158d-4566-83fe-1a643cf17344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.000789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbde8e8-158d-4566-83fe-1a643cf17344 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.001074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbde8e8-158d-4566-83fe-1a643cf17344 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.001349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbde8e8-158d-4566-83fe-1a643cf17344 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.002188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbde8e8-158d-4566-83fe-1a643cf17344 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:42.002554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbde8e8-158d-4566-83fe-1a643cf17344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.004013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbde8e8-158d-4566-83fe-1a643cf17344 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.006423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:42.015298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082e91a-03ee-4c8f-bf5c-e78b858c054d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.028973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 335/667] 10.222.0.17 () {56 vars in 1139 bytes} [Sun Jul 24 19:41:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:42.028973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbde8e8-158d-4566-83fe-1a643cf17344 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.059638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb4758-6fcd-4051-a5cf-d4df48439711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.060439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb4758-6fcd-4051-a5cf-d4df48439711 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.060750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb4758-6fcd-4051-a5cf-d4df48439711 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.061028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb4758-6fcd-4051-a5cf-d4df48439711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.076215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb4758-6fcd-4051-a5cf-d4df48439711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.076215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb4758-6fcd-4051-a5cf-d4df48439711 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-9ceb4758-6fcd-4051-a5cf-d4df48439711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:42.091467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:42.092391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:42.093714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:42.093714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:42.093714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:42.093714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:42.093714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 333/668] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:42.093714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb4758-6fcd-4051-a5cf-d4df48439711 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.116817 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:42.136602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbde8e8-158d-4566-83fe-1a643cf17344 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.173479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798796c8-fc5a-45d0-9b9f-587a9eecdd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.174151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798796c8-fc5a-45d0-9b9f-587a9eecdd26 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.174437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798796c8-fc5a-45d0-9b9f-587a9eecdd26 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.174716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798796c8-fc5a-45d0-9b9f-587a9eecdd26 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.175458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798796c8-fc5a-45d0-9b9f-587a9eecdd26 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:42.175768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798796c8-fc5a-45d0-9b9f-587a9eecdd26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.177280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798796c8-fc5a-45d0-9b9f-587a9eecdd26 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.185984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 336/669] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:42.191156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798796c8-fc5a-45d0-9b9f-587a9eecdd26 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.203276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb4758-6fcd-4051-a5cf-d4df48439711 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.238426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9b9f8-07d3-471c-8bf4-969e3700517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.239192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9b9f8-07d3-471c-8bf4-969e3700517b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.239481 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9b9f8-07d3-471c-8bf4-969e3700517b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.239783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9b9f8-07d3-471c-8bf4-969e3700517b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.240713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9b9f8-07d3-471c-8bf4-969e3700517b None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=123b7f659c6a4ceba18ce268b1335542) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:42.241033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9b9f8-07d3-471c-8bf4-969e3700517b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.242573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9b9f8-07d3-471c-8bf4-969e3700517b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.249793 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 334/670] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/role_assignments?user.id=123b7f659c6a4ceba18ce268b1335542&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:42.250705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9b9f8-07d3-471c-8bf4-969e3700517b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.273167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:42.322347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798796c8-fc5a-45d0-9b9f-587a9eecdd26 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.363140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab099bfa-db7a-4395-a344-88376a11a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.364394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab099bfa-db7a-4395-a344-88376a11a75b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.364926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab099bfa-db7a-4395-a344-88376a11a75b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.365385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab099bfa-db7a-4395-a344-88376a11a75b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.381189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:42.407480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9b9f8-07d3-471c-8bf4-969e3700517b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.451043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c30eb-b5fd-453b-b2a8-3fae8cee7c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.451559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c30eb-b5fd-453b-b2a8-3fae8cee7c84 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.451970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c30eb-b5fd-453b-b2a8-3fae8cee7c84 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.452352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c30eb-b5fd-453b-b2a8-3fae8cee7c84 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.453223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c30eb-b5fd-453b-b2a8-3fae8cee7c84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:42.453604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c30eb-b5fd-453b-b2a8-3fae8cee7c84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.454987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c30eb-b5fd-453b-b2a8-3fae8cee7c84 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.456093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab099bfa-db7a-4395-a344-88376a11a75b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.457811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab099bfa-db7a-4395-a344-88376a11a75b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.469624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 335/671] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:42.470579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c30eb-b5fd-453b-b2a8-3fae8cee7c84 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.555231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:42.578157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 337/672] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:42.579721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab099bfa-db7a-4395-a344-88376a11a75b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.594862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c30eb-b5fd-453b-b2a8-3fae8cee7c84 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.630192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:42.633831 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736dc51-8952-4323-9e51-9da1e2acecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.634678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736dc51-8952-4323-9e51-9da1e2acecfe None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.634963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736dc51-8952-4323-9e51-9da1e2acecfe None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.635244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736dc51-8952-4323-9e51-9da1e2acecfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.646159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736dc51-8952-4323-9e51-9da1e2acecfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.647629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736dc51-8952-4323-9e51-9da1e2acecfe None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a736dc51-8952-4323-9e51-9da1e2acecfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:42.671230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:42.672384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:42.673518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:42.673518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:42.673518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:42.673518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:42.677530 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 336/673] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:42.678713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736dc51-8952-4323-9e51-9da1e2acecfe None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.697135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab099bfa-db7a-4395-a344-88376a11a75b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.734743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2787ac-aac1-4ac0-b765-e96ebacbfc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.735796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2787ac-aac1-4ac0-b765-e96ebacbfc3e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.736229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2787ac-aac1-4ac0-b765-e96ebacbfc3e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.736706 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2787ac-aac1-4ac0-b765-e96ebacbfc3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.744984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2787ac-aac1-4ac0-b765-e96ebacbfc3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.746688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2787ac-aac1-4ac0-b765-e96ebacbfc3e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-4a2787ac-aac1-4ac0-b765-e96ebacbfc3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:42.761340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:42.762437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:42.763483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:42.763483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:42.763483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:42.763483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:42.763483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:42.763483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:42.763483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:42.763483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:42.765512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 338/674] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:42.766421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2787ac-aac1-4ac0-b765-e96ebacbfc3e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.794824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736dc51-8952-4323-9e51-9da1e2acecfe None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.817379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:42.832537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118549d3-2635-4eb8-848e-7d3de39c1cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.838147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118549d3-2635-4eb8-848e-7d3de39c1cf7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.838147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118549d3-2635-4eb8-848e-7d3de39c1cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.838147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118549d3-2635-4eb8-848e-7d3de39c1cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.846684 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118549d3-2635-4eb8-848e-7d3de39c1cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.848356 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118549d3-2635-4eb8-848e-7d3de39c1cf7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-118549d3-2635-4eb8-848e-7d3de39c1cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:42.869213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:42.870220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:42.871208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:42.871208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:42.871208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:42.871208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:42.873251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 337/675] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:42.874240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118549d3-2635-4eb8-848e-7d3de39c1cf7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.883386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2787ac-aac1-4ac0-b765-e96ebacbfc3e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:42.884906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:42.928419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c944f-fe4e-48a9-bea0-6146d558a5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:42.929141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c944f-fe4e-48a9-bea0-6146d558a5e4 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:42.929425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c944f-fe4e-48a9-bea0-6146d558a5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:42.929953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c944f-fe4e-48a9-bea0-6146d558a5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:42.930517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c944f-fe4e-48a9-bea0-6146d558a5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:42.930827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c944f-fe4e-48a9-bea0-6146d558a5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:42.932204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c944f-fe4e-48a9-bea0-6146d558a5e4 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:42.950647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 339/676] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:42.950647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c944f-fe4e-48a9-bea0-6146d558a5e4 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:42.988981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118549d3-2635-4eb8-848e-7d3de39c1cf7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.025911 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2327a-ddc2-46fa-828b-dca2dc02311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.026647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2327a-ddc2-46fa-828b-dca2dc02311d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.026846 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2327a-ddc2-46fa-828b-dca2dc02311d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.027122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2327a-ddc2-46fa-828b-dca2dc02311d None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.027904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2327a-ddc2-46fa-828b-dca2dc02311d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:43.028254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2327a-ddc2-46fa-828b-dca2dc02311d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.029906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2327a-ddc2-46fa-828b-dca2dc02311d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.046508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 338/677] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:43.046508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2327a-ddc2-46fa-828b-dca2dc02311d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:43.064915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c944f-fe4e-48a9-bea0-6146d558a5e4 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.080143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:43.101615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb7a1b-7294-431a-a9ec-49c94dcd7e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.102306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb7a1b-7294-431a-a9ec-49c94dcd7e7f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.102596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb7a1b-7294-431a-a9ec-49c94dcd7e7f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.102874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb7a1b-7294-431a-a9ec-49c94dcd7e7f None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.103645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb7a1b-7294-431a-a9ec-49c94dcd7e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:43.103958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb7a1b-7294-431a-a9ec-49c94dcd7e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.105387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb7a1b-7294-431a-a9ec-49c94dcd7e7f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.117065 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 340/678] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:43.118467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6154c-0573-4af0-b316-804a44673bd8 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.118794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6154c-0573-4af0-b316-804a44673bd8 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.119080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6154c-0573-4af0-b316-804a44673bd8 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.120478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 341/679] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:43.125310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6154c-0573-4af0-b316-804a44673bd8 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:43.151326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:43.161214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2327a-ddc2-46fa-828b-dca2dc02311d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.202507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c47856-d7f3-49f4-a850-3a2631b6a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.203785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c47856-d7f3-49f4-a850-3a2631b6a3df None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.204136 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c47856-d7f3-49f4-a850-3a2631b6a3df None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.204478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c47856-d7f3-49f4-a850-3a2631b6a3df None admin] PATH_INFO: `/v3/users/swift` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.214732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c47856-d7f3-49f4-a850-3a2631b6a3df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.216653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c47856-d7f3-49f4-a850-3a2631b6a3df None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.228674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6154c-0573-4af0-b316-804a44673bd8 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-53c47856-d7f3-49f4-a850-3a2631b6a3df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:43.233913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:43.235024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:43.236079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:43.241876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 339/680] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:43.241876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c47856-d7f3-49f4-a850-3a2631b6a3df None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:43.263303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cac7c-b411-4b9b-befa-4813e2780aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.263979 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cac7c-b411-4b9b-befa-4813e2780aeb None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.264276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cac7c-b411-4b9b-befa-4813e2780aeb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.264558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cac7c-b411-4b9b-befa-4813e2780aeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.265991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cac7c-b411-4b9b-befa-4813e2780aeb None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=bca33ef49ba44832a7fb45b2317d211c) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:43.265991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cac7c-b411-4b9b-befa-4813e2780aeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.267187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cac7c-b411-4b9b-befa-4813e2780aeb None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.273490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 342/681] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:43 2022] GET /identity/v3/role_assignments?user.id=bca33ef49ba44832a7fb45b2317d211c&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:43.275570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a915b0-27c9-4cd4-9d95-343da535ed5e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.275870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a915b0-27c9-4cd4-9d95-343da535ed5e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.276151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a915b0-27c9-4cd4-9d95-343da535ed5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.293688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-92a915b0-27c9-4cd4-9d95-343da535ed5e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:43.294931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-92a915b0-27c9-4cd4-9d95-343da535ed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:43.366432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:43.376949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 343/682] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:43.377522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c47856-d7f3-49f4-a850-3a2631b6a3df None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.378347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a915b0-27c9-4cd4-9d95-343da535ed5e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:43.414340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:43.418756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e6e7e-ee15-4927-8537-6da6647ec78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.419596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e6e7e-ee15-4927-8537-6da6647ec78f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.419906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e6e7e-ee15-4927-8537-6da6647ec78f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.424526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e6e7e-ee15-4927-8537-6da6647ec78f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.428020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e6e7e-ee15-4927-8537-6da6647ec78f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.429632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e6e7e-ee15-4927-8537-6da6647ec78f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-1f5e6e7e-ee15-4927-8537-6da6647ec78f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:43.445385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:43.446615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:43.448264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:43.448264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:43.448264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:43.448264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:43.448264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:43.448264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:43.448264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:43.448264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:43.452660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 340/683] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:43.453808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e6e7e-ee15-4927-8537-6da6647ec78f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:43.522281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a915b0-27c9-4cd4-9d95-343da535ed5e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.569852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847d995-e6a9-4fcd-9423-dacfa6304292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.570706 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847d995-e6a9-4fcd-9423-dacfa6304292 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.571091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847d995-e6a9-4fcd-9423-dacfa6304292 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.571487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847d995-e6a9-4fcd-9423-dacfa6304292 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.572821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847d995-e6a9-4fcd-9423-dacfa6304292 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:43.573294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847d995-e6a9-4fcd-9423-dacfa6304292 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.574976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847d995-e6a9-4fcd-9423-dacfa6304292 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.585334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e6e7e-ee15-4927-8537-6da6647ec78f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.597096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 344/684] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Jul 24 19:41:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:43.598059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847d995-e6a9-4fcd-9423-dacfa6304292 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:43.635198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b55486-10ee-4508-97e2-44cf8a8ee9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.636084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b55486-10ee-4508-97e2-44cf8a8ee9b0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.636480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b55486-10ee-4508-97e2-44cf8a8ee9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.636852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b55486-10ee-4508-97e2-44cf8a8ee9b0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.637658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b55486-10ee-4508-97e2-44cf8a8ee9b0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.639055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b55486-10ee-4508-97e2-44cf8a8ee9b0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-83b55486-10ee-4508-97e2-44cf8a8ee9b0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:43.655329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:43.656419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:43.657303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 341/685] 10.222.0.17 () {56 vars in 1129 bytes} [Sun Jul 24 19:41:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:43.658172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b55486-10ee-4508-97e2-44cf8a8ee9b0 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:43.679788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:43.689366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:43.720867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847d995-e6a9-4fcd-9423-dacfa6304292 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.770856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b278832-de07-4b89-b86a-e41ba8b6500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.771627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b278832-de07-4b89-b86a-e41ba8b6500b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.771991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b278832-de07-4b89-b86a-e41ba8b6500b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.772398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b278832-de07-4b89-b86a-e41ba8b6500b None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.772816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b55486-10ee-4508-97e2-44cf8a8ee9b0 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.773409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b278832-de07-4b89-b86a-e41ba8b6500b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:43.773791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b278832-de07-4b89-b86a-e41ba8b6500b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.776802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b278832-de07-4b89-b86a-e41ba8b6500b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.795399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 345/686] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:43.796464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b278832-de07-4b89-b86a-e41ba8b6500b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:43.813151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf8175-b88d-4f4a-b5f5-20bc385297ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.814087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8175-b88d-4f4a-b5f5-20bc385297ce None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.814550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8175-b88d-4f4a-b5f5-20bc385297ce None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.815026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8175-b88d-4f4a-b5f5-20bc385297ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.887817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf8175-b88d-4f4a-b5f5-20bc385297ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.889654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf8175-b88d-4f4a-b5f5-20bc385297ce None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.911510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:43.933532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b278832-de07-4b89-b86a-e41ba8b6500b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:43.972551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a7218-21e8-4ae8-9ff6-6214db97f3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:43.973714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a7218-21e8-4ae8-9ff6-6214db97f3f8 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:43.974198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a7218-21e8-4ae8-9ff6-6214db97f3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:43.974655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a7218-21e8-4ae8-9ff6-6214db97f3f8 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:43.975538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a7218-21e8-4ae8-9ff6-6214db97f3f8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:43.976026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a7218-21e8-4ae8-9ff6-6214db97f3f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:43.977785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a7218-21e8-4ae8-9ff6-6214db97f3f8 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:43.979160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:43.984188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 346/687] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Jul 24 19:41:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:43.985514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a7218-21e8-4ae8-9ff6-6214db97f3f8 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:44.016730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 342/688] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:44.017990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf8175-b88d-4f4a-b5f5-20bc385297ce None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:44.129214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf8175-b88d-4f4a-b5f5-20bc385297ce None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:44.130053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a7218-21e8-4ae8-9ff6-6214db97f3f8 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:44.149761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:44.171608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3c34e-53ef-4081-94e6-18aa6153bf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:44.172390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3c34e-53ef-4081-94e6-18aa6153bf3a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:44.172709 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3c34e-53ef-4081-94e6-18aa6153bf3a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:44.173006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3c34e-53ef-4081-94e6-18aa6153bf3a None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:44.173845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3c34e-53ef-4081-94e6-18aa6153bf3a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:44.174222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3c34e-53ef-4081-94e6-18aa6153bf3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:44.175907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3c34e-53ef-4081-94e6-18aa6153bf3a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:44.177045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2fb5-270f-49d7-a2b2-368955abc84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:44.177947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2fb5-270f-49d7-a2b2-368955abc84c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:44.178426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2fb5-270f-49d7-a2b2-368955abc84c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:44.178893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2fb5-270f-49d7-a2b2-368955abc84c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:44.179942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b2fb5-270f-49d7-a2b2-368955abc84c None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=b9763f9899214df3929f3816f009e857) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:44.180462 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b2fb5-270f-49d7-a2b2-368955abc84c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:44.183255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 343/689] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Jul 24 19:41:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:44.183760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b2fb5-270f-49d7-a2b2-368955abc84c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:44.189725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3c34e-53ef-4081-94e6-18aa6153bf3a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:44.200284 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 347/690] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:43 2022] GET /identity/v3/role_assignments?user.id=b9763f9899214df3929f3816f009e857&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:44.359129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3c34e-53ef-4081-94e6-18aa6153bf3a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:44.401824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df173b06-4006-43f6-8ca5-41a4359ac6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:44.402996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df173b06-4006-43f6-8ca5-41a4359ac6b3 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:44.407085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df173b06-4006-43f6-8ca5-41a4359ac6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:44.407741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df173b06-4006-43f6-8ca5-41a4359ac6b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:44.420581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df173b06-4006-43f6-8ca5-41a4359ac6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:44.424179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df173b06-4006-43f6-8ca5-41a4359ac6b3 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-df173b06-4006-43f6-8ca5-41a4359ac6b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:44.441897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:44.442830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:44.443733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:44.443733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:44.443733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:44.443733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:44.443733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 344/691] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:44.449470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2fb5-270f-49d7-a2b2-368955abc84c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:44.510973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf67e-7859-45d1-9ffe-b8fdd1ea4702 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:44.511294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf67e-7859-45d1-9ffe-b8fdd1ea4702 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:44.511533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf67e-7859-45d1-9ffe-b8fdd1ea4702 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:44.513176 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 345/692] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:44.518119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98985d97-49cf-480e-a940-d2f3f3d7e24b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:44.518410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98985d97-49cf-480e-a940-d2f3f3d7e24b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:44.518692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98985d97-49cf-480e-a940-d2f3f3d7e24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:44.534069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:44.535836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-98985d97-49cf-480e-a940-d2f3f3d7e24b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:44.537087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-98985d97-49cf-480e-a940-d2f3f3d7e24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:44.577281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2fb5-270f-49d7-a2b2-368955abc84c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:44.585036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:44.612894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f5764-a204-4f0e-847c-0d63b7eb8e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:44.613567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f5764-a204-4f0e-847c-0d63b7eb8e91 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:44.613857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f5764-a204-4f0e-847c-0d63b7eb8e91 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:44.614126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f5764-a204-4f0e-847c-0d63b7eb8e91 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:44.614850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f5764-a204-4f0e-847c-0d63b7eb8e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:44.615156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f5764-a204-4f0e-847c-0d63b7eb8e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:44.616727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f5764-a204-4f0e-847c-0d63b7eb8e91 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:44.630558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 348/693] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:44.636533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f5764-a204-4f0e-847c-0d63b7eb8e91 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:44.675432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 346/694] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:44.746195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98985d97-49cf-480e-a940-d2f3f3d7e24b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:44.814601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f5764-a204-4f0e-847c-0d63b7eb8e91 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:44.861025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:44.887150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dbbf4-6c7d-4c26-b58e-20c2fd2e125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:44.887150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dbbf4-6c7d-4c26-b58e-20c2fd2e125b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:44.887448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dbbf4-6c7d-4c26-b58e-20c2fd2e125b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:44.887746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dbbf4-6c7d-4c26-b58e-20c2fd2e125b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:44.901861 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002dbbf4-6c7d-4c26-b58e-20c2fd2e125b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:44.905130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002dbbf4-6c7d-4c26-b58e-20c2fd2e125b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:44.916085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98985d97-49cf-480e-a940-d2f3f3d7e24b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-002dbbf4-6c7d-4c26-b58e-20c2fd2e125b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:44.935602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:44.936929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:44.937798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:44.937798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:44.937798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:44.937798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:44.937798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:44.937798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:44.937798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:44.937798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:44.937798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 349/695] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:44.943191 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dbbf4-6c7d-4c26-b58e-20c2fd2e125b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:44.976098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3b993-23de-4485-80ec-b185588a42aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:44.977322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3b993-23de-4485-80ec-b185588a42aa None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:44.977804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3b993-23de-4485-80ec-b185588a42aa None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:44.978876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3b993-23de-4485-80ec-b185588a42aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:45.001297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3b993-23de-4485-80ec-b185588a42aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-e0c3b993-23de-4485-80ec-b185588a42aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:45.012489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:45.013449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:45.014290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:45.017659 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 347/696] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:45.023563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3b993-23de-4485-80ec-b185588a42aa None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:45.053029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:45.155951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3b993-23de-4485-80ec-b185588a42aa None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:45.186601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dbbf4-6c7d-4c26-b58e-20c2fd2e125b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:45.227251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a83da-0275-4607-b134-eef12953b4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:45.227931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a83da-0275-4607-b134-eef12953b4e9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:45.228827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a83da-0275-4607-b134-eef12953b4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:45.229109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a83da-0275-4607-b134-eef12953b4e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:45.229867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a83da-0275-4607-b134-eef12953b4e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:45.230176 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a83da-0275-4607-b134-eef12953b4e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:45.231554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a83da-0275-4607-b134-eef12953b4e9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:45.249462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 348/697] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:45.255334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a83da-0275-4607-b134-eef12953b4e9 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:45.265343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a308cc-898b-40e7-ac1e-a195da504acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:45.266027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a308cc-898b-40e7-ac1e-a195da504acd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:45.266317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a308cc-898b-40e7-ac1e-a195da504acd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:45.266597 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a308cc-898b-40e7-ac1e-a195da504acd None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:45.267400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a308cc-898b-40e7-ac1e-a195da504acd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:45.267712 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a308cc-898b-40e7-ac1e-a195da504acd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:45.269179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a308cc-898b-40e7-ac1e-a195da504acd None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:45.293426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 350/698] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:45.306783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a308cc-898b-40e7-ac1e-a195da504acd None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:45.359348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:45.371330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a83da-0275-4607-b134-eef12953b4e9 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:45.417073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:45.421048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee237cd-6c82-4b3e-bae1-5a4c868232f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:45.424546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee237cd-6c82-4b3e-bae1-5a4c868232f5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:45.424546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee237cd-6c82-4b3e-bae1-5a4c868232f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:45.424546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee237cd-6c82-4b3e-bae1-5a4c868232f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:45.447792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee237cd-6c82-4b3e-bae1-5a4c868232f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:45.449281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee237cd-6c82-4b3e-bae1-5a4c868232f5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-1ee237cd-6c82-4b3e-bae1-5a4c868232f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:45.486210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:45.487268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:45.488242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:45.488242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:45.488242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:45.488242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:45.491023 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 349/699] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:45.497058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee237cd-6c82-4b3e-bae1-5a4c868232f5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:45.512577 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a308cc-898b-40e7-ac1e-a195da504acd None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:45.567032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f33753-0a4e-46d7-aaa5-8e95c91ace9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:45.567707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f33753-0a4e-46d7-aaa5-8e95c91ace9c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:45.567994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f33753-0a4e-46d7-aaa5-8e95c91ace9c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:45.568265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f33753-0a4e-46d7-aaa5-8e95c91ace9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:45.569136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f33753-0a4e-46d7-aaa5-8e95c91ace9c None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=0849f31902da4613babb10a76c4548f3) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:45.569447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f33753-0a4e-46d7-aaa5-8e95c91ace9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:45.570880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f33753-0a4e-46d7-aaa5-8e95c91ace9c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:45.584223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 351/700] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:45 2022] GET /identity/v3/role_assignments?user.id=0849f31902da4613babb10a76c4548f3&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:45.643968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee237cd-6c82-4b3e-bae1-5a4c868232f5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:45.697841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8241069-bff1-4b96-b2e7-337d29f2f152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:45.698516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8241069-bff1-4b96-b2e7-337d29f2f152 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:45.698863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8241069-bff1-4b96-b2e7-337d29f2f152 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:45.699142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8241069-bff1-4b96-b2e7-337d29f2f152 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:45.699927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8241069-bff1-4b96-b2e7-337d29f2f152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:45.700239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8241069-bff1-4b96-b2e7-337d29f2f152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:45.701635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8241069-bff1-4b96-b2e7-337d29f2f152 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:45.707727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:45.721085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 350/701] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:45.726480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f33753-0a4e-46d7-aaa5-8e95c91ace9c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:45.773616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422a381-2939-4417-b492-bfff18b0d5bb None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:45.773912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422a381-2939-4417-b492-bfff18b0d5bb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:45.774094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422a381-2939-4417-b492-bfff18b0d5bb None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:45.775628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 351/702] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:45.781513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313864b-ce16-4baa-be37-c739e0145481 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:45.781825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313864b-ce16-4baa-be37-c739e0145481 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:45.782091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313864b-ce16-4baa-be37-c739e0145481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:45.800156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-9313864b-ce16-4baa-be37-c739e0145481 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:45.801428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9313864b-ce16-4baa-be37-c739e0145481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:45.878086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 352/703] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:45.906218 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313864b-ce16-4baa-be37-c739e0145481 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:45.941545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f33753-0a4e-46d7-aaa5-8e95c91ace9c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:45.954130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:46.000132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae17b6-a793-45a3-85f1-59456a998269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:46.001874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae17b6-a793-45a3-85f1-59456a998269 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.002309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae17b6-a793-45a3-85f1-59456a998269 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.002879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae17b6-a793-45a3-85f1-59456a998269 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.016157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae17b6-a793-45a3-85f1-59456a998269 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:46.018236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae17b6-a793-45a3-85f1-59456a998269 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-1dae17b6-a793-45a3-85f1-59456a998269 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:46.039593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:46.041382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:46.042636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 352/704] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Jul 24 19:41:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:46.049668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae17b6-a793-45a3-85f1-59456a998269 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:46.065273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313864b-ce16-4baa-be37-c739e0145481 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:46.102830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:46.114037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ef1d0-d064-4d13-aec2-867e57018e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:46.115171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ef1d0-d064-4d13-aec2-867e57018e73 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.115526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ef1d0-d064-4d13-aec2-867e57018e73 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.119355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ef1d0-d064-4d13-aec2-867e57018e73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.131287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ef1d0-d064-4d13-aec2-867e57018e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:46.132794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ef1d0-d064-4d13-aec2-867e57018e73 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-802ef1d0-d064-4d13-aec2-867e57018e73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:46.153811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:46.154754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:46.155628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:46.155628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:46.155628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:46.155628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:46.155628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 353/705] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:46.161538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ef1d0-d064-4d13-aec2-867e57018e73 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:46.197264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae17b6-a793-45a3-85f1-59456a998269 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:46.249399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2f679-ad25-4c50-88fd-7da4cdfe7eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:46.251364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2f679-ad25-4c50-88fd-7da4cdfe7eca None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.251853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2f679-ad25-4c50-88fd-7da4cdfe7eca None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.252349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2f679-ad25-4c50-88fd-7da4cdfe7eca None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.253244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2f679-ad25-4c50-88fd-7da4cdfe7eca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:46.253558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2f679-ad25-4c50-88fd-7da4cdfe7eca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:46.254992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2f679-ad25-4c50-88fd-7da4cdfe7eca None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:46.280412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 353/706] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Jul 24 19:41:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:46.286421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ef1d0-d064-4d13-aec2-867e57018e73 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:46.286573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2f679-ad25-4c50-88fd-7da4cdfe7eca None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:46.315797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:46.328304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3a224-9ac4-432b-909c-bce9119259a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:46.335995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3a224-9ac4-432b-909c-bce9119259a0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.335995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3a224-9ac4-432b-909c-bce9119259a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.335995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3a224-9ac4-432b-909c-bce9119259a0 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.335995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3a224-9ac4-432b-909c-bce9119259a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:46.335995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3a224-9ac4-432b-909c-bce9119259a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:46.335995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3a224-9ac4-432b-909c-bce9119259a0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:46.350586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 354/707] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:46.358025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3a224-9ac4-432b-909c-bce9119259a0 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:46.445300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:46.451986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2f679-ad25-4c50-88fd-7da4cdfe7eca None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:46.515961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598d0aa-3457-496a-8797-5ff9694be292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:46.516635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598d0aa-3457-496a-8797-5ff9694be292 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.516998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598d0aa-3457-496a-8797-5ff9694be292 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.517287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598d0aa-3457-496a-8797-5ff9694be292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.583295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3a224-9ac4-432b-909c-bce9119259a0 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:46.597731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598d0aa-3457-496a-8797-5ff9694be292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:46.599266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598d0aa-3457-496a-8797-5ff9694be292 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:46.627797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:46.669475 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644cffe-06f5-48bd-9eef-089d54096fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:46.670187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644cffe-06f5-48bd-9eef-089d54096fc5 None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.670481 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644cffe-06f5-48bd-9eef-089d54096fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.670763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644cffe-06f5-48bd-9eef-089d54096fc5 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.671487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644cffe-06f5-48bd-9eef-089d54096fc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:46.672990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644cffe-06f5-48bd-9eef-089d54096fc5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:46.712150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 354/708] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 24 19:41:46.720885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598d0aa-3457-496a-8797-5ff9694be292 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:46.788670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 355/709] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:41:46 2022] POST /identity/v3/users => generated 254 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:46.788670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1550e2-e481-439d-8ca7-98b9e27c6776 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.788670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1550e2-e481-439d-8ca7-98b9e27c6776 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.788670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1550e2-e481-439d-8ca7-98b9e27c6776 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.788670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 356/710] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:46.806505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caaa88-f816-4b28-b8da-2daf9d5506ad None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.806817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caaa88-f816-4b28-b8da-2daf9d5506ad None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.807068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caaa88-f816-4b28-b8da-2daf9d5506ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.831740 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-02caaa88-f816-4b28-b8da-2daf9d5506ad None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:46.833964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-02caaa88-f816-4b28-b8da-2daf9d5506ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:46.853335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598d0aa-3457-496a-8797-5ff9694be292 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:46.883586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:46.896128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c9f11-e7c9-4811-80ba-dccae245fa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:46.897209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c9f11-e7c9-4811-80ba-dccae245fa87 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.897593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c9f11-e7c9-4811-80ba-dccae245fa87 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.898001 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c9f11-e7c9-4811-80ba-dccae245fa87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.911488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c9f11-e7c9-4811-80ba-dccae245fa87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:46.913449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c9f11-e7c9-4811-80ba-dccae245fa87 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:46.932534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 357/711] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:46.933336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad94019-5502-43c3-855e-9dcaeb4071af None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.933459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad94019-5502-43c3-855e-9dcaeb4071af None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.934971 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad94019-5502-43c3-855e-9dcaeb4071af None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-1c3c9f11-e7c9-4811-80ba-dccae245fa87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:46.935087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:46.937145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:46.939140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:46.939140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:46.939140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:46.939140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:46.939602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 358/712] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:46.944759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 355/713] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:46.945349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef55676-8d10-4aa1-8508-97c4982bd683 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:46.945349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef55676-8d10-4aa1-8508-97c4982bd683 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:46.945349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef55676-8d10-4aa1-8508-97c4982bd683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:46.951337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c9f11-e7c9-4811-80ba-dccae245fa87 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:46.964783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1ef55676-8d10-4aa1-8508-97c4982bd683 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:46.966329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1ef55676-8d10-4aa1-8508-97c4982bd683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:46.990806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:47.059739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 359/714] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:47.059739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef55676-8d10-4aa1-8508-97c4982bd683 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:47.091387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c9f11-e7c9-4811-80ba-dccae245fa87 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:47.129481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822c8b6-74f4-4ad5-b353-c396a7688508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:47.130310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822c8b6-74f4-4ad5-b353-c396a7688508 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:47.130599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822c8b6-74f4-4ad5-b353-c396a7688508 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:47.131159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822c8b6-74f4-4ad5-b353-c396a7688508 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:47.131899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822c8b6-74f4-4ad5-b353-c396a7688508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:47.132209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822c8b6-74f4-4ad5-b353-c396a7688508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:47.133599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822c8b6-74f4-4ad5-b353-c396a7688508 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:47.148758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 356/715] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:47.149664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822c8b6-74f4-4ad5-b353-c396a7688508 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:47.187718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:47.194946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef55676-8d10-4aa1-8508-97c4982bd683 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:47.240476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d831f2-51c1-491e-adbc-a71d3c2efb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:47.241196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d831f2-51c1-491e-adbc-a71d3c2efb2e None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:47.241480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d831f2-51c1-491e-adbc-a71d3c2efb2e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:47.241753 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d831f2-51c1-491e-adbc-a71d3c2efb2e None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:47.242480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d831f2-51c1-491e-adbc-a71d3c2efb2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:47.244706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d831f2-51c1-491e-adbc-a71d3c2efb2e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:47.256384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822c8b6-74f4-4ad5-b353-c396a7688508 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:47.295482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18249ea1-17e7-4e1e-adb5-dd1cca229b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:47.296339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18249ea1-17e7-4e1e-adb5-dd1cca229b24 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:47.296663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18249ea1-17e7-4e1e-adb5-dd1cca229b24 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:47.296933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18249ea1-17e7-4e1e-adb5-dd1cca229b24 None admin] PATH_INFO: `/v3/services/network` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:47.297579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18249ea1-17e7-4e1e-adb5-dd1cca229b24 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:47.298936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18249ea1-17e7-4e1e-adb5-dd1cca229b24 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-18249ea1-17e7-4e1e-adb5-dd1cca229b24 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:47.313443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:47.314538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 24 19:41:47.315430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:47.318974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 357/716] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:47 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:47.320139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18249ea1-17e7-4e1e-adb5-dd1cca229b24 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:47.353024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 360/717] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:41:47 2022] POST /identity/v3/services => generated 251 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:47.353024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d831f2-51c1-491e-adbc-a71d3c2efb2e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:47.387678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:47.517556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:47.521223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18249ea1-17e7-4e1e-adb5-dd1cca229b24 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:47.572385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8205a-2eb3-493b-9896-4c3f8c225e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:47.573233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8205a-2eb3-493b-9896-4c3f8c225e27 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:47.573519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8205a-2eb3-493b-9896-4c3f8c225e27 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:47.573795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8205a-2eb3-493b-9896-4c3f8c225e27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:47.580264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d831f2-51c1-491e-adbc-a71d3c2efb2e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:47.583313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8205a-2eb3-493b-9896-4c3f8c225e27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:47.585674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8205a-2eb3-493b-9896-4c3f8c225e27 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-88e8205a-2eb3-493b-9896-4c3f8c225e27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:47.609851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:47.610793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:47.611689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:47.611689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:47.611689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:47.611689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:47.611689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:47.611689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:47.611689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:47.611689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:47.611689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 358/718] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:47.616668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8205a-2eb3-493b-9896-4c3f8c225e27 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:47.627565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef943d1d-4057-4c0f-a858-8f60d665d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:47.628251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef943d1d-4057-4c0f-a858-8f60d665d3ca None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:47.628541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef943d1d-4057-4c0f-a858-8f60d665d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:47.628869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef943d1d-4057-4c0f-a858-8f60d665d3ca None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:47.629585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef943d1d-4057-4c0f-a858-8f60d665d3ca None admin] RBAC: Adding query filter params (name=network) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:47.629896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef943d1d-4057-4c0f-a858-8f60d665d3ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:47.631392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef943d1d-4057-4c0f-a858-8f60d665d3ca None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:47.638290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 361/719] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:41:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:47.652799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef943d1d-4057-4c0f-a858-8f60d665d3ca None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:47.734889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8205a-2eb3-493b-9896-4c3f8c225e27 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:47.738552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:47.778054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef943d1d-4057-4c0f-a858-8f60d665d3ca None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:47.778549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483f800-91d8-4b0d-967d-606f0fec9a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:47.779530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483f800-91d8-4b0d-967d-606f0fec9a01 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:47.780035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483f800-91d8-4b0d-967d-606f0fec9a01 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:47.780575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483f800-91d8-4b0d-967d-606f0fec9a01 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:47.781602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483f800-91d8-4b0d-967d-606f0fec9a01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:47.782106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483f800-91d8-4b0d-967d-606f0fec9a01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:47.783728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483f800-91d8-4b0d-967d-606f0fec9a01 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:47.798522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 359/720] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:47.809917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483f800-91d8-4b0d-967d-606f0fec9a01 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:47.828670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e8107-a608-4436-a713-02fa173b832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:47.828670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8107-a608-4436-a713-02fa173b832b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:47.828670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8107-a608-4436-a713-02fa173b832b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:47.828670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8107-a608-4436-a713-02fa173b832b None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:47.829280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e8107-a608-4436-a713-02fa173b832b None admin] RBAC: Adding query filter params (type=network) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:47.829494 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e8107-a608-4436-a713-02fa173b832b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:47.830867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e8107-a608-4436-a713-02fa173b832b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:47.841107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 362/721] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:41:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:47.856249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:47.997837 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483f800-91d8-4b0d-967d-606f0fec9a01 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:48.045784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae72e2d-f6fa-4c3b-98b4-8edfab0ecc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:48.046809 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae72e2d-f6fa-4c3b-98b4-8edfab0ecc73 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:48.047297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae72e2d-f6fa-4c3b-98b4-8edfab0ecc73 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:48.047807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae72e2d-f6fa-4c3b-98b4-8edfab0ecc73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:48.049000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae72e2d-f6fa-4c3b-98b4-8edfab0ecc73 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=123b7f659c6a4ceba18ce268b1335542) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:48.049517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae72e2d-f6fa-4c3b-98b4-8edfab0ecc73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:48.051143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae72e2d-f6fa-4c3b-98b4-8edfab0ecc73 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:48.059094 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 360/722] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:47 2022] GET /identity/v3/role_assignments?user.id=123b7f659c6a4ceba18ce268b1335542&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:48.541103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8f017-231b-4ca3-bbc2-6145c8c6c9f8 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:48.541879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8f017-231b-4ca3-bbc2-6145c8c6c9f8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:48.541954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8f017-231b-4ca3-bbc2-6145c8c6c9f8 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:48.543586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 363/723] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:48.548892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437e94-d46c-44e9-bd58-93834ae464f7 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:48.549187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437e94-d46c-44e9-bd58-93834ae464f7 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:48.549465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437e94-d46c-44e9-bd58-93834ae464f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:48.566795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-af437e94-d46c-44e9-bd58-93834ae464f7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:48.567992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-af437e94-d46c-44e9-bd58-93834ae464f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:48.662392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 361/724] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:48.689637 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8f017-231b-4ca3-bbc2-6145c8c6c9f8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:48.846302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8f017-231b-4ca3-bbc2-6145c8c6c9f8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:48.891187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcf488-c0da-4c30-8a67-53f45a1673b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:48.892107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcf488-c0da-4c30-8a67-53f45a1673b8 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:48.892457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcf488-c0da-4c30-8a67-53f45a1673b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:48.892829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcf488-c0da-4c30-8a67-53f45a1673b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:48.904185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:48.908942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcf488-c0da-4c30-8a67-53f45a1673b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-b2bcf488-c0da-4c30-8a67-53f45a1673b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:48.917780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:48.918893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:48.919737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 364/725] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:48.929780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af437e94-d46c-44e9-bd58-93834ae464f7 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:48.952691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:49.116627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af437e94-d46c-44e9-bd58-93834ae464f7 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:49.161660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b74b1e-7343-4da7-9d26-fce411a19a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:49.162728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b74b1e-7343-4da7-9d26-fce411a19a5f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:49.163211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b74b1e-7343-4da7-9d26-fce411a19a5f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:49.163687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b74b1e-7343-4da7-9d26-fce411a19a5f None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:49.164682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b74b1e-7343-4da7-9d26-fce411a19a5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:49.165187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b74b1e-7343-4da7-9d26-fce411a19a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:49.170060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b74b1e-7343-4da7-9d26-fce411a19a5f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:49.184309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 362/726] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:49.189980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcf488-c0da-4c30-8a67-53f45a1673b8 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:49.334987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcf488-c0da-4c30-8a67-53f45a1673b8 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:49.383705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b558ba5-f1a2-4b17-832c-cbd097f4f3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:49.384548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b558ba5-f1a2-4b17-832c-cbd097f4f3b2 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:49.384853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b558ba5-f1a2-4b17-832c-cbd097f4f3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:49.385130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b558ba5-f1a2-4b17-832c-cbd097f4f3b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:49.399313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b558ba5-f1a2-4b17-832c-cbd097f4f3b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:49.401392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b558ba5-f1a2-4b17-832c-cbd097f4f3b2 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-2b558ba5-f1a2-4b17-832c-cbd097f4f3b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:49.428334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:49.429710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:49.431195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:49.431195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:49.431195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:49.431195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:49.436852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 365/727] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:49.438678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b74b1e-7343-4da7-9d26-fce411a19a5f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:49.487678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f87d-f958-4827-93b6-8565cd478c35 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:49.488062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f87d-f958-4827-93b6-8565cd478c35 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:49.488275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f87d-f958-4827-93b6-8565cd478c35 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:49.489875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 366/728] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:49.504781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ba844-6629-4e2b-8c40-5d68e5e6aedb None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:49.505309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ba844-6629-4e2b-8c40-5d68e5e6aedb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:49.505699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ba844-6629-4e2b-8c40-5d68e5e6aedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:49.539140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-fe6ba844-6629-4e2b-8c40-5d68e5e6aedb None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:49.540400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fe6ba844-6629-4e2b-8c40-5d68e5e6aedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:49.586285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:49.618556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b74b1e-7343-4da7-9d26-fce411a19a5f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:49.660721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:49.682216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d994932-2fa6-465a-b48e-2f749dff3584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:49.683663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d994932-2fa6-465a-b48e-2f749dff3584 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:49.684379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d994932-2fa6-465a-b48e-2f749dff3584 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:49.685176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d994932-2fa6-465a-b48e-2f749dff3584 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:49.686598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d994932-2fa6-465a-b48e-2f749dff3584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:49.687520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d994932-2fa6-465a-b48e-2f749dff3584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:49.691754 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d994932-2fa6-465a-b48e-2f749dff3584 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:49.724226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 363/729] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:49.726451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 367/730] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 24 19:41:49.733652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ba844-6629-4e2b-8c40-5d68e5e6aedb None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:49.752048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d994932-2fa6-465a-b48e-2f749dff3584 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:49.879419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ba844-6629-4e2b-8c40-5d68e5e6aedb None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:49.960785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46c01b-c68d-42cf-80fd-93b364f409d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:49.961976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46c01b-c68d-42cf-80fd-93b364f409d7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:49.963055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46c01b-c68d-42cf-80fd-93b364f409d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:49.963536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46c01b-c68d-42cf-80fd-93b364f409d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:49.972577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46c01b-c68d-42cf-80fd-93b364f409d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:49.974410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46c01b-c68d-42cf-80fd-93b364f409d7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:49.974594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d994932-2fa6-465a-b48e-2f749dff3584 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-ee46c01b-c68d-42cf-80fd-93b364f409d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:49.992343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:49.993388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:49.994403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:49.997282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 368/731] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:50.003171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46c01b-c68d-42cf-80fd-93b364f409d7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:50.027975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:50.045365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e1c73-17c4-4399-ac8f-9dae23712448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:50.046194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e1c73-17c4-4399-ac8f-9dae23712448 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.046483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e1c73-17c4-4399-ac8f-9dae23712448 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.046762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e1c73-17c4-4399-ac8f-9dae23712448 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.052925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:50.066055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e1c73-17c4-4399-ac8f-9dae23712448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-0c4e1c73-17c4-4399-ac8f-9dae23712448 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:50.075916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:50.081264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:50.082123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:50.085623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 364/732] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:50.093184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e1c73-17c4-4399-ac8f-9dae23712448 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:50.133511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46c01b-c68d-42cf-80fd-93b364f409d7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:50.171997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72c7b6-54c5-46e7-8736-561e90a98619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:50.172708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72c7b6-54c5-46e7-8736-561e90a98619 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.172998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72c7b6-54c5-46e7-8736-561e90a98619 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.173275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72c7b6-54c5-46e7-8736-561e90a98619 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.174123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72c7b6-54c5-46e7-8736-561e90a98619 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:50.174431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72c7b6-54c5-46e7-8736-561e90a98619 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:50.175876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72c7b6-54c5-46e7-8736-561e90a98619 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:50.214828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 369/733] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Jul 24 19:41:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:50.222062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72c7b6-54c5-46e7-8736-561e90a98619 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:50.249905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e1c73-17c4-4399-ac8f-9dae23712448 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:50.293040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b92845-17db-4242-a29f-5cd130ec7ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:50.293863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b92845-17db-4242-a29f-5cd130ec7ff9 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.294249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b92845-17db-4242-a29f-5cd130ec7ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.294626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b92845-17db-4242-a29f-5cd130ec7ff9 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.295500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b92845-17db-4242-a29f-5cd130ec7ff9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:50.295910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b92845-17db-4242-a29f-5cd130ec7ff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:50.297545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b92845-17db-4242-a29f-5cd130ec7ff9 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:50.306674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 365/734] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:50.308202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44f4d1-fdda-409d-8e5f-4b73e9896d52 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.308594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44f4d1-fdda-409d-8e5f-4b73e9896d52 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.308997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44f4d1-fdda-409d-8e5f-4b73e9896d52 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.310624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 366/735] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:50.312103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44f4d1-fdda-409d-8e5f-4b73e9896d52 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:50.343778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:50.347994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72c7b6-54c5-46e7-8736-561e90a98619 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:50.378941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:50.386996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fe76a-bd46-47d3-8987-ab86d6356e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:50.387705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fe76a-bd46-47d3-8987-ab86d6356e6b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.388102 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fe76a-bd46-47d3-8987-ab86d6356e6b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.388574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fe76a-bd46-47d3-8987-ab86d6356e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.444908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44f4d1-fdda-409d-8e5f-4b73e9896d52 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:50.463665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fe76a-bd46-47d3-8987-ab86d6356e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:50.465173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fe76a-bd46-47d3-8987-ab86d6356e6b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:50.481929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c35bfb-3611-4ad9-99ab-fde0757a8e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:50.482728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c35bfb-3611-4ad9-99ab-fde0757a8e80 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.483017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c35bfb-3611-4ad9-99ab-fde0757a8e80 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.483293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c35bfb-3611-4ad9-99ab-fde0757a8e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.494292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c35bfb-3611-4ad9-99ab-fde0757a8e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:50.495735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c35bfb-3611-4ad9-99ab-fde0757a8e80 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-13c35bfb-3611-4ad9-99ab-fde0757a8e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:50.514778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:50.515711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:50.516863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:50.516863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:50.516863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:50.516863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:50.516863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 367/736] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:50.517987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac5488-d044-4b4f-9d8c-c3659495daf4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.518234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac5488-d044-4b4f-9d8c-c3659495daf4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.518521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac5488-d044-4b4f-9d8c-c3659495daf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.536769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-93ac5488-d044-4b4f-9d8c-c3659495daf4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:50.537962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-93ac5488-d044-4b4f-9d8c-c3659495daf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:50.575131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:50.581068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 370/737] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:50 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 24 19:41:50.586053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8bd15-cf38-4db0-9637-919907d23a86 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.586345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8bd15-cf38-4db0-9637-919907d23a86 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.592860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8bd15-cf38-4db0-9637-919907d23a86 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.592860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 371/738] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:50.592860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8bd15-cf38-4db0-9637-919907d23a86 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:50.613769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 368/739] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:50.615006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac5488-d044-4b4f-9d8c-c3659495daf4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:50.655273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:50.706145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8bd15-cf38-4db0-9637-919907d23a86 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:50.720596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac5488-d044-4b4f-9d8c-c3659495daf4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:50.740552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34e2d7-2b56-4c49-a8bc-7fa341c34dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:50.741254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34e2d7-2b56-4c49-a8bc-7fa341c34dbc None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.741540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34e2d7-2b56-4c49-a8bc-7fa341c34dbc None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.741814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34e2d7-2b56-4c49-a8bc-7fa341c34dbc None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.742529 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34e2d7-2b56-4c49-a8bc-7fa341c34dbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:50.742835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34e2d7-2b56-4c49-a8bc-7fa341c34dbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:50.744805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34e2d7-2b56-4c49-a8bc-7fa341c34dbc None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:50.756349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0f79e-e7f3-4b87-9243-6441159eba53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:50.757972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 372/740] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:50.758091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0f79e-e7f3-4b87-9243-6441159eba53 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.758091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0f79e-e7f3-4b87-9243-6441159eba53 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.758091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0f79e-e7f3-4b87-9243-6441159eba53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.758300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cc257-4f7d-4314-bbbf-ac09a46365b2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.758300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cc257-4f7d-4314-bbbf-ac09a46365b2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.758516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cc257-4f7d-4314-bbbf-ac09a46365b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.770959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0f79e-e7f3-4b87-9243-6441159eba53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:50.772413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0f79e-e7f3-4b87-9243-6441159eba53 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:50.775164 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-cf8cc257-4f7d-4314-bbbf-ac09a46365b2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:50.776350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-cf8cc257-4f7d-4314-bbbf-ac09a46365b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-58c0f79e-e7f3-4b87-9243-6441159eba53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:50.789944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:50.791343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:50.792514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:50.792514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:50.792514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:50.792514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:50.792514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 369/741] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:50.792830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0f79e-e7f3-4b87-9243-6441159eba53 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:50.830535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:50.843582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 373/742] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:50.845038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3440-2335-4cc5-900a-38ebeb50b9b1 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.845404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3440-2335-4cc5-900a-38ebeb50b9b1 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.845690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3440-2335-4cc5-900a-38ebeb50b9b1 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.847100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 374/743] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:50.847654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3440-2335-4cc5-900a-38ebeb50b9b1 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:50.885939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:50.897319 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0f79e-e7f3-4b87-9243-6441159eba53 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:50.932237 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfb235-b9b2-48ea-8a20-03288844973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:50.933002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfb235-b9b2-48ea-8a20-03288844973d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.933292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfb235-b9b2-48ea-8a20-03288844973d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.933571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfb235-b9b2-48ea-8a20-03288844973d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:50.934230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfb235-b9b2-48ea-8a20-03288844973d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:50.935600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfb235-b9b2-48ea-8a20-03288844973d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-5ebfb235-b9b2-48ea-8a20-03288844973d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:50.947862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:50.948971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:50.949784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 370/744] 10.222.0.17 () {56 vars in 1129 bytes} [Sun Jul 24 19:41:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:50.950851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfb235-b9b2-48ea-8a20-03288844973d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:50.958219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3440-2335-4cc5-900a-38ebeb50b9b1 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:50.996695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6d49c-c044-4365-b753-4004e590142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:50.997501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6d49c-c044-4365-b753-4004e590142a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:50.997783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6d49c-c044-4365-b753-4004e590142a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:50.998055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6d49c-c044-4365-b753-4004e590142a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.005704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6d49c-c044-4365-b753-4004e590142a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.007144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6d49c-c044-4365-b753-4004e590142a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-31a6d49c-c044-4365-b753-4004e590142a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:51.021908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:51.022877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:51.023799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 375/745] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:41:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:51.025462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e3bec-a9d9-4ba1-91bf-6670f80432ce None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.025462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e3bec-a9d9-4ba1-91bf-6670f80432ce None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.025462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e3bec-a9d9-4ba1-91bf-6670f80432ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.042998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-696e3bec-a9d9-4ba1-91bf-6670f80432ce None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:51.044169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-696e3bec-a9d9-4ba1-91bf-6670f80432ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:51.066385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfb235-b9b2-48ea-8a20-03288844973d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.087053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:51.100420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3807736-1ad7-4e38-982f-fafb6545a0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.101113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3807736-1ad7-4e38-982f-fafb6545a0b4 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.101443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3807736-1ad7-4e38-982f-fafb6545a0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.101730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3807736-1ad7-4e38-982f-fafb6545a0b4 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.102454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3807736-1ad7-4e38-982f-fafb6545a0b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:51.102820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3807736-1ad7-4e38-982f-fafb6545a0b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.113824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3807736-1ad7-4e38-982f-fafb6545a0b4 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.113944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 376/746] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 24 19:41:51.113944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e3bec-a9d9-4ba1-91bf-6670f80432ce None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:51.125217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 371/747] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:51.126296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3807736-1ad7-4e38-982f-fafb6545a0b4 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:51.141682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:51.217740 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e3bec-a9d9-4ba1-91bf-6670f80432ce None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.226893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3807736-1ad7-4e38-982f-fafb6545a0b4 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.251788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38258aa-19b3-4e51-8b79-90f5d4f98d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.252584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38258aa-19b3-4e51-8b79-90f5d4f98d6f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.252897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38258aa-19b3-4e51-8b79-90f5d4f98d6f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.253176 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38258aa-19b3-4e51-8b79-90f5d4f98d6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.260937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4540c-2916-4e68-a61b-7b08741bcd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.261607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4540c-2916-4e68-a61b-7b08741bcd97 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.261881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4540c-2916-4e68-a61b-7b08741bcd97 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.262156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4540c-2916-4e68-a61b-7b08741bcd97 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.262859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4540c-2916-4e68-a61b-7b08741bcd97 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:51.263163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4540c-2916-4e68-a61b-7b08741bcd97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.264182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38258aa-19b3-4e51-8b79-90f5d4f98d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:51.264529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4540c-2916-4e68-a61b-7b08741bcd97 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.268851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 372/748] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Jul 24 19:41:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:51.269617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4540c-2916-4e68-a61b-7b08741bcd97 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a38258aa-19b3-4e51-8b79-90f5d4f98d6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:51.271934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:51.273223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:51.274434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 377/749] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:51.276084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38258aa-19b3-4e51-8b79-90f5d4f98d6f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:51.316046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:51.347949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:51.370581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4540c-2916-4e68-a61b-7b08741bcd97 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.374909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38258aa-19b3-4e51-8b79-90f5d4f98d6f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.407731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1c237-9018-4521-9c62-fe36f561a95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.408412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1c237-9018-4521-9c62-fe36f561a95d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.408718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1c237-9018-4521-9c62-fe36f561a95d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.408999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1c237-9018-4521-9c62-fe36f561a95d None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.409292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c432dc-7f3e-4df3-8f32-dc7f49163478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.409848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1c237-9018-4521-9c62-fe36f561a95d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:51.410092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c432dc-7f3e-4df3-8f32-dc7f49163478 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.410184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1c237-9018-4521-9c62-fe36f561a95d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.410383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c432dc-7f3e-4df3-8f32-dc7f49163478 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.410659 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c432dc-7f3e-4df3-8f32-dc7f49163478 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.411575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1c237-9018-4521-9c62-fe36f561a95d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.418063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c432dc-7f3e-4df3-8f32-dc7f49163478 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.423275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c432dc-7f3e-4df3-8f32-dc7f49163478 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.430267 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 373/750] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Jul 24 19:41:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:51.431128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1c237-9018-4521-9c62-fe36f561a95d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-e4c432dc-7f3e-4df3-8f32-dc7f49163478 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:51.443378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:51.444916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 378/751] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:51.447158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c432dc-7f3e-4df3-8f32-dc7f49163478 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:51.534374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1c237-9018-4521-9c62-fe36f561a95d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.543502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:51.547026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c432dc-7f3e-4df3-8f32-dc7f49163478 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.568205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7106ad-546e-4a46-b03b-780f950139ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.568898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7106ad-546e-4a46-b03b-780f950139ff None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.569184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7106ad-546e-4a46-b03b-780f950139ff None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.569461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7106ad-546e-4a46-b03b-780f950139ff None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.570187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7106ad-546e-4a46-b03b-780f950139ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.572060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7106ad-546e-4a46-b03b-780f950139ff None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.574343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:51.580606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b5dcf-f5d8-4749-b4be-5278e2a30b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.581296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b5dcf-f5d8-4749-b4be-5278e2a30b02 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.581580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b5dcf-f5d8-4749-b4be-5278e2a30b02 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.581855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b5dcf-f5d8-4749-b4be-5278e2a30b02 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.582558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b5dcf-f5d8-4749-b4be-5278e2a30b02 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:51.582865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b5dcf-f5d8-4749-b4be-5278e2a30b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.584210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b5dcf-f5d8-4749-b4be-5278e2a30b02 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.589490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 379/752] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Jul 24 19:41:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:51.590263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b5dcf-f5d8-4749-b4be-5278e2a30b02 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:51.654521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 374/753] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:41:51 2022] POST /identity/v3/services => generated 240 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:51.655385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7106ad-546e-4a46-b03b-780f950139ff None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:51.695578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b5dcf-f5d8-4749-b4be-5278e2a30b02 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.741692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0061bf-a46c-4ff9-81d3-a92dac1877b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.744175 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0061bf-a46c-4ff9-81d3-a92dac1877b8 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.744502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0061bf-a46c-4ff9-81d3-a92dac1877b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.744840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0061bf-a46c-4ff9-81d3-a92dac1877b8 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.745691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0061bf-a46c-4ff9-81d3-a92dac1877b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:51.746031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0061bf-a46c-4ff9-81d3-a92dac1877b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.747742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0061bf-a46c-4ff9-81d3-a92dac1877b8 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.762764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 380/754] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:51.762764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0061bf-a46c-4ff9-81d3-a92dac1877b8 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:51.785142 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7106ad-546e-4a46-b03b-780f950139ff None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.785561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:51.823659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bd5ec-8b2d-4374-9f17-9394fdfbd090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.824443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bd5ec-8b2d-4374-9f17-9394fdfbd090 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.824852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bd5ec-8b2d-4374-9f17-9394fdfbd090 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.825140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bd5ec-8b2d-4374-9f17-9394fdfbd090 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.874141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0061bf-a46c-4ff9-81d3-a92dac1877b8 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:51.901232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:51.908598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bd5ec-8b2d-4374-9f17-9394fdfbd090 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.910078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bd5ec-8b2d-4374-9f17-9394fdfbd090 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.913837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab15311-7587-4574-a978-0e2de4a28657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:51.917716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab15311-7587-4574-a978-0e2de4a28657 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:51.917716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab15311-7587-4574-a978-0e2de4a28657 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:51.917716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab15311-7587-4574-a978-0e2de4a28657 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:51.917716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab15311-7587-4574-a978-0e2de4a28657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:51.917716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab15311-7587-4574-a978-0e2de4a28657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:51.918429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab15311-7587-4574-a978-0e2de4a28657 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:51.936679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 381/755] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:51.936679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab15311-7587-4574-a978-0e2de4a28657 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:52.031310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:52.044973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 375/756] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:41:52.044973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bd5ec-8b2d-4374-9f17-9394fdfbd090 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:52.057889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab15311-7587-4574-a978-0e2de4a28657 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:52.097060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86eb8f4-0b9d-4cef-b113-526e10e37b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:52.097720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86eb8f4-0b9d-4cef-b113-526e10e37b45 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:52.098008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86eb8f4-0b9d-4cef-b113-526e10e37b45 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:52.098288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86eb8f4-0b9d-4cef-b113-526e10e37b45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:52.099077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86eb8f4-0b9d-4cef-b113-526e10e37b45 None admin] RBAC: Adding query filter params (interface=public, service_id=47b9f3564cd7454c8861476c9fc3d721, region_id=RegionOne) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:52.099391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86eb8f4-0b9d-4cef-b113-526e10e37b45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:52.100798 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86eb8f4-0b9d-4cef-b113-526e10e37b45 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:52.108980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 382/757] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Jul 24 19:41:51 2022] GET /identity/v3/endpoints?service_id=47b9f3564cd7454c8861476c9fc3d721&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:52.109812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86eb8f4-0b9d-4cef-b113-526e10e37b45 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:52.160439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:52.168926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bd5ec-8b2d-4374-9f17-9394fdfbd090 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:52.209650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46ad28-5278-41b2-8d8e-85c9f40c30ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:52.210447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46ad28-5278-41b2-8d8e-85c9f40c30ef None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:52.210728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46ad28-5278-41b2-8d8e-85c9f40c30ef None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:52.211062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46ad28-5278-41b2-8d8e-85c9f40c30ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:52.222208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46ad28-5278-41b2-8d8e-85c9f40c30ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:52.223764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46ad28-5278-41b2-8d8e-85c9f40c30ef None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:52.239195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86eb8f4-0b9d-4cef-b113-526e10e37b45 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-3a46ad28-5278-41b2-8d8e-85c9f40c30ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:52.242799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:52.244245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:52.250143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:52.250143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:52.250143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:52.250143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:52.250143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 376/758] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:52.250143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46ad28-5278-41b2-8d8e-85c9f40c30ef None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:52.279589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da797c05-2749-4469-8ebf-0b207ab8cfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:52.280244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da797c05-2749-4469-8ebf-0b207ab8cfe1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:52.280528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da797c05-2749-4469-8ebf-0b207ab8cfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:52.280833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da797c05-2749-4469-8ebf-0b207ab8cfe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:52.281780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da797c05-2749-4469-8ebf-0b207ab8cfe1 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=0849f31902da4613babb10a76c4548f3) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:52.282203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da797c05-2749-4469-8ebf-0b207ab8cfe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:52.283683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da797c05-2749-4469-8ebf-0b207ab8cfe1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:52.300765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 383/759] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:52 2022] GET /identity/v3/role_assignments?user.id=0849f31902da4613babb10a76c4548f3&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:52.300765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da797c05-2749-4469-8ebf-0b207ab8cfe1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:52.313754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:52.374108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46ad28-5278-41b2-8d8e-85c9f40c30ef None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:52.426619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfddf6-adb4-48e0-a6c2-51a12e5df670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:52.427453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfddf6-adb4-48e0-a6c2-51a12e5df670 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:52.427737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfddf6-adb4-48e0-a6c2-51a12e5df670 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:52.428016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfddf6-adb4-48e0-a6c2-51a12e5df670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:52.442465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfddf6-adb4-48e0-a6c2-51a12e5df670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:52.443933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfddf6-adb4-48e0-a6c2-51a12e5df670 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:52.451932 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da797c05-2749-4469-8ebf-0b207ab8cfe1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:52.458422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-4adfddf6-adb4-48e0-a6c2-51a12e5df670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:52.472987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:52.480587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:52.481735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:52.481735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:52.481735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:52.481735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:52.481735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 377/760] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:52.481735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfddf6-adb4-48e0-a6c2-51a12e5df670 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:52.501691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3437f7b-0a08-4734-ab64-30536669cbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:52.502428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3437f7b-0a08-4734-ab64-30536669cbaa None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:52.502718 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3437f7b-0a08-4734-ab64-30536669cbaa None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:52.503002 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3437f7b-0a08-4734-ab64-30536669cbaa None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:52.503737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3437f7b-0a08-4734-ab64-30536669cbaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:52.506076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3437f7b-0a08-4734-ab64-30536669cbaa None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:52.513436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 384/761] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Jul 24 19:41:52 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:52.521633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3437f7b-0a08-4734-ab64-30536669cbaa None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:52.601881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfddf6-adb4-48e0-a6c2-51a12e5df670 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:52.645513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd76f9e-5ab4-48fc-b0ea-4d05f860767a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:52.646602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd76f9e-5ab4-48fc-b0ea-4d05f860767a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:52.647138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd76f9e-5ab4-48fc-b0ea-4d05f860767a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:52.647727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd76f9e-5ab4-48fc-b0ea-4d05f860767a None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:52.648756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd76f9e-5ab4-48fc-b0ea-4d05f860767a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:52.649368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd76f9e-5ab4-48fc-b0ea-4d05f860767a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:52.651711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd76f9e-5ab4-48fc-b0ea-4d05f860767a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:52.667235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 378/762] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:52.673461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd76f9e-5ab4-48fc-b0ea-4d05f860767a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:52.695472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:52.748144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:52.758281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3437f7b-0a08-4734-ab64-30536669cbaa None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:52.803878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd76f9e-5ab4-48fc-b0ea-4d05f860767a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:52.811305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4810c51-11a7-4c8c-8c78-4ee5d0f769cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:52.812955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4810c51-11a7-4c8c-8c78-4ee5d0f769cb None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:52.813475 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4810c51-11a7-4c8c-8c78-4ee5d0f769cb None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:52.813977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4810c51-11a7-4c8c-8c78-4ee5d0f769cb None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:52.815378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4810c51-11a7-4c8c-8c78-4ee5d0f769cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:52.816299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4810c51-11a7-4c8c-8c78-4ee5d0f769cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:52.818228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4810c51-11a7-4c8c-8c78-4ee5d0f769cb None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:52.840671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 385/763] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:52.849723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4810c51-11a7-4c8c-8c78-4ee5d0f769cb None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:52.857962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3b4dc-8025-4ea4-b840-859d987f8686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:52.860515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3b4dc-8025-4ea4-b840-859d987f8686 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:52.864042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3b4dc-8025-4ea4-b840-859d987f8686 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:52.864663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3b4dc-8025-4ea4-b840-859d987f8686 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:52.885927 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3b4dc-8025-4ea4-b840-859d987f8686 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:52.891962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3b4dc-8025-4ea4-b840-859d987f8686 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-a2c3b4dc-8025-4ea4-b840-859d987f8686 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:52.915749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:52.917081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:52.918016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:52.920948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 379/764] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Jul 24 19:41:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:52.926205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3b4dc-8025-4ea4-b840-859d987f8686 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:53.001444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4810c51-11a7-4c8c-8c78-4ee5d0f769cb None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:53.034833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:53.046964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a71a5-2ca2-419d-b630-47e68c822715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:53.047159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a71a5-2ca2-419d-b630-47e68c822715 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:53.048367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a71a5-2ca2-419d-b630-47e68c822715 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:53.048367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a71a5-2ca2-419d-b630-47e68c822715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:53.060721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a71a5-2ca2-419d-b630-47e68c822715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:53.062261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a71a5-2ca2-419d-b630-47e68c822715 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-061a71a5-2ca2-419d-b630-47e68c822715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:53.095059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:53.096040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:53.097387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:53.097387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:53.097387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:53.097387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:53.097387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:53.097387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:53.097387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:53.097387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:53.097387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 386/765] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:53.104336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a71a5-2ca2-419d-b630-47e68c822715 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:53.173206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3b4dc-8025-4ea4-b840-859d987f8686 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:53.207504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:53.255384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce66f76-4549-457c-b397-06ea605a4d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:53.256131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce66f76-4549-457c-b397-06ea605a4d1d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:53.256422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce66f76-4549-457c-b397-06ea605a4d1d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:53.256722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce66f76-4549-457c-b397-06ea605a4d1d None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:53.257586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce66f76-4549-457c-b397-06ea605a4d1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:53.257908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce66f76-4549-457c-b397-06ea605a4d1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:53.259948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce66f76-4549-457c-b397-06ea605a4d1d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:53.266119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a71a5-2ca2-419d-b630-47e68c822715 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:53.304882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 380/766] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Jul 24 19:41:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:53.306384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90942a45-e279-452d-b91c-022c3a7c22c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:53.307062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942a45-e279-452d-b91c-022c3a7c22c5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:53.307344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942a45-e279-452d-b91c-022c3a7c22c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:53.307619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942a45-e279-452d-b91c-022c3a7c22c5 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:53.308419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90942a45-e279-452d-b91c-022c3a7c22c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:53.308758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90942a45-e279-452d-b91c-022c3a7c22c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:53.310226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90942a45-e279-452d-b91c-022c3a7c22c5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:53.310613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce66f76-4549-457c-b397-06ea605a4d1d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:53.327128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 387/767] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:53.333232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90942a45-e279-452d-b91c-022c3a7c22c5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:53.401147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:53.465396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90942a45-e279-452d-b91c-022c3a7c22c5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:53.507880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e35a5-6825-4934-9460-6cb3b6318189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:53.508547 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e35a5-6825-4934-9460-6cb3b6318189 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:53.508886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e35a5-6825-4934-9460-6cb3b6318189 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:53.509167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e35a5-6825-4934-9460-6cb3b6318189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:53.510579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e35a5-6825-4934-9460-6cb3b6318189 None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=8fb7202aa94640129f33a310f6d00876) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:53.510898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e35a5-6825-4934-9460-6cb3b6318189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:53.512497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e35a5-6825-4934-9460-6cb3b6318189 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:53.519863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 388/768] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:41:53 2022] GET /identity/v3/role_assignments?user.id=8fb7202aa94640129f33a310f6d00876&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:53.560345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce66f76-4549-457c-b397-06ea605a4d1d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:53.622574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8baad2-997f-4c62-ba38-ef763310387f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:53.623767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8baad2-997f-4c62-ba38-ef763310387f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:53.624281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8baad2-997f-4c62-ba38-ef763310387f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:53.624787 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8baad2-997f-4c62-ba38-ef763310387f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:53.641364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8baad2-997f-4c62-ba38-ef763310387f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:53.643201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8baad2-997f-4c62-ba38-ef763310387f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-bb8baad2-997f-4c62-ba38-ef763310387f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:53.664192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:53.665768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:53.666921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:53.666921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:53.666921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:53.666921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:53.666921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 381/769] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:53.673947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e35a5-6825-4934-9460-6cb3b6318189 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:53.801790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e35a5-6825-4934-9460-6cb3b6318189 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:53.819687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:53.838148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f7be7-eb65-490a-aec2-3c85fe957777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:53.838822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f7be7-eb65-490a-aec2-3c85fe957777 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:53.839105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f7be7-eb65-490a-aec2-3c85fe957777 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:53.839380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f7be7-eb65-490a-aec2-3c85fe957777 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:53.840112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f7be7-eb65-490a-aec2-3c85fe957777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:53.840420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f7be7-eb65-490a-aec2-3c85fe957777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:53.841833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f7be7-eb65-490a-aec2-3c85fe957777 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:53.855883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 389/770] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:53.861201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8baad2-997f-4c62-ba38-ef763310387f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:53.892388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:53.969479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8baad2-997f-4c62-ba38-ef763310387f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:54.002644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25593f07-43f3-4353-a07a-5d6f3c3cac32 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.002974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25593f07-43f3-4353-a07a-5d6f3c3cac32 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.003202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25593f07-43f3-4353-a07a-5d6f3c3cac32 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.004569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 390/771] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:41:54.010220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8fe20-c5d9-4051-87be-c964086930ac None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.010508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8fe20-c5d9-4051-87be-c964086930ac None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.010786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8fe20-c5d9-4051-87be-c964086930ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.020384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ba9ab-babe-44d2-a1d8-039a01c83408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:54.021650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba9ab-babe-44d2-a1d8-039a01c83408 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.022166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba9ab-babe-44d2-a1d8-039a01c83408 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.022648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba9ab-babe-44d2-a1d8-039a01c83408 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.029104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-43a8fe20-c5d9-4051-87be-c964086930ac None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:54.030306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-43a8fe20-c5d9-4051-87be-c964086930ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:54.042282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ba9ab-babe-44d2-a1d8-039a01c83408 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:54.044120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ba9ab-babe-44d2-a1d8-039a01c83408 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-7f3ba9ab-babe-44d2-a1d8-039a01c83408 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:54.069047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:54.069953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:54.070830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:54.070830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:54.070830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:54.070830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:54.070830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:54.070830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:54.070830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:54.070830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:54.077195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 382/772] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:54.082789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ba9ab-babe-44d2-a1d8-039a01c83408 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:54.110248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 391/773] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:41:54.146655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8fe20-c5d9-4051-87be-c964086930ac None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:54.245014 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:54.260099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8fe20-c5d9-4051-87be-c964086930ac None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:54.268298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:54.279500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ba9ab-babe-44d2-a1d8-039a01c83408 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:54.299079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fb195-2b52-4a0b-a197-db32d03c9330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:54.299935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fb195-2b52-4a0b-a197-db32d03c9330 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.300225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fb195-2b52-4a0b-a197-db32d03c9330 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.300501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fb195-2b52-4a0b-a197-db32d03c9330 None admin] PATH_INFO: `/v3/services/network` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.301199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fb195-2b52-4a0b-a197-db32d03c9330 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:54.302585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fb195-2b52-4a0b-a197-db32d03c9330 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-9d5fb195-2b52-4a0b-a197-db32d03c9330 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:54.315628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:54.316633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:54.318090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 392/774] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:54 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:54.323127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fb195-2b52-4a0b-a197-db32d03c9330 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:54.366030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979347f-9c44-4551-a668-b78687c544da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:54.366715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979347f-9c44-4551-a668-b78687c544da None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.367007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979347f-9c44-4551-a668-b78687c544da None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.367290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979347f-9c44-4551-a668-b78687c544da None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.368089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979347f-9c44-4551-a668-b78687c544da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:54.368438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979347f-9c44-4551-a668-b78687c544da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:54.369903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979347f-9c44-4551-a668-b78687c544da None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:54.382796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 383/775] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:54.389004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979347f-9c44-4551-a668-b78687c544da None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:54.442636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fb195-2b52-4a0b-a197-db32d03c9330 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:54.495559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979347f-9c44-4551-a668-b78687c544da None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:54.522010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd25ca8-2f23-4bb1-9331-2ce49fdc7966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:54.522844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd25ca8-2f23-4bb1-9331-2ce49fdc7966 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.523131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd25ca8-2f23-4bb1-9331-2ce49fdc7966 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.523409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd25ca8-2f23-4bb1-9331-2ce49fdc7966 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.524120 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd25ca8-2f23-4bb1-9331-2ce49fdc7966 None admin] RBAC: Adding query filter params (name=network) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:54.525077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd25ca8-2f23-4bb1-9331-2ce49fdc7966 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:54.526395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd25ca8-2f23-4bb1-9331-2ce49fdc7966 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:54.533259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 393/776] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:41:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:54.538197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd25ca8-2f23-4bb1-9331-2ce49fdc7966 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:54.544776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dabd5f-a0be-4afd-845b-ad8c1f31c8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:54.545690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dabd5f-a0be-4afd-845b-ad8c1f31c8bd None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.545972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dabd5f-a0be-4afd-845b-ad8c1f31c8bd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.546249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dabd5f-a0be-4afd-845b-ad8c1f31c8bd None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/123b7f659c6a4ceba18ce268b1335542/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.554010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dabd5f-a0be-4afd-845b-ad8c1f31c8bd None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=123b7f659c6a4ceba18ce268b1335542, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:54.555464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dabd5f-a0be-4afd-845b-ad8c1f31c8bd None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:54.588008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:54.637394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 384/777] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:41:54 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/123b7f659c6a4ceba18ce268b1335542/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:54.672066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd25ca8-2f23-4bb1-9331-2ce49fdc7966 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:54.724178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79c46f-f9d0-49a3-8d90-4327b0e831db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:54.724428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455982f-097b-48ae-ae59-e56db7191a7f None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.724894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79c46f-f9d0-49a3-8d90-4327b0e831db None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.725181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79c46f-f9d0-49a3-8d90-4327b0e831db None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.725452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455982f-097b-48ae-ae59-e56db7191a7f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.725554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79c46f-f9d0-49a3-8d90-4327b0e831db None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.726028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455982f-097b-48ae-ae59-e56db7191a7f None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.726195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79c46f-f9d0-49a3-8d90-4327b0e831db None admin] RBAC: Adding query filter params (type=network) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:54.726497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79c46f-f9d0-49a3-8d90-4327b0e831db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:54.727975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79c46f-f9d0-49a3-8d90-4327b0e831db None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:54.728631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 385/778] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:54.734649 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91aac1d-109c-45c8-998e-bcfa620c6548 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.735280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91aac1d-109c-45c8-998e-bcfa620c6548 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.735755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91aac1d-109c-45c8-998e-bcfa620c6548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.736720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 394/779] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:41:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:54.744962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79c46f-f9d0-49a3-8d90-4327b0e831db None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:54.766934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f91aac1d-109c-45c8-998e-bcfa620c6548 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:54.769116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f91aac1d-109c-45c8-998e-bcfa620c6548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:54.771554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:54.901620 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 386/780] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:41:54.914315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:54.918713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79c46f-f9d0-49a3-8d90-4327b0e831db None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:54.926369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91aac1d-109c-45c8-998e-bcfa620c6548 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:54.963380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217b8f9-70c2-4e38-9442-a39a4cf53d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:54.964524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217b8f9-70c2-4e38-9442-a39a4cf53d18 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:54.964997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217b8f9-70c2-4e38-9442-a39a4cf53d18 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:54.966377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217b8f9-70c2-4e38-9442-a39a4cf53d18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:54.967420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217b8f9-70c2-4e38-9442-a39a4cf53d18 None admin] RBAC: Adding query filter params (interface=public, service_id=7d977d9c95ce4fd4aa9a1d7d2f175241, region_id=RegionOne) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:54.967893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217b8f9-70c2-4e38-9442-a39a4cf53d18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:54.969495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217b8f9-70c2-4e38-9442-a39a4cf53d18 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:54.978240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 395/781] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Jul 24 19:41:54 2022] GET /identity/v3/endpoints?service_id=7d977d9c95ce4fd4aa9a1d7d2f175241&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:54.984149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217b8f9-70c2-4e38-9442-a39a4cf53d18 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:55.043218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91aac1d-109c-45c8-998e-bcfa620c6548 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:55.082206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d25b-c96f-43ad-ad22-daa4d3a01f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:55.083393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d25b-c96f-43ad-ad22-daa4d3a01f67 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:55.083972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d25b-c96f-43ad-ad22-daa4d3a01f67 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:55.084444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d25b-c96f-43ad-ad22-daa4d3a01f67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:55.097051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:55.101131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217b8f9-70c2-4e38-9442-a39a4cf53d18 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:55.108976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d25b-c96f-43ad-ad22-daa4d3a01f67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-8224d25b-c96f-43ad-ad22-daa4d3a01f67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:55.122923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:55.123890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:55.124766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:55.131347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 387/782] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:55.136163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d25b-c96f-43ad-ad22-daa4d3a01f67 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:55.204447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f1cec-a7db-4fff-8d01-00360ae8772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:55.205539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f1cec-a7db-4fff-8d01-00360ae8772d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:55.206031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f1cec-a7db-4fff-8d01-00360ae8772d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:55.206526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f1cec-a7db-4fff-8d01-00360ae8772d None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:55.207439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f1cec-a7db-4fff-8d01-00360ae8772d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:55.209040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f1cec-a7db-4fff-8d01-00360ae8772d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:55.223844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 396/783] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Jul 24 19:41:54 2022] GET /identity/v3/services => generated 772 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:55.267598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d25b-c96f-43ad-ad22-daa4d3a01f67 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:55.296432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9a3dc-1156-4cae-95e0-be1242d4c5e8 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:55.297135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9a3dc-1156-4cae-95e0-be1242d4c5e8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:55.297678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9a3dc-1156-4cae-95e0-be1242d4c5e8 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:55.299750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 397/784] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:41:55.305553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6136fa4-84ff-4c70-8aa2-b31ad2e76813 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:55.306673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6136fa4-84ff-4c70-8aa2-b31ad2e76813 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:55.307189 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6136fa4-84ff-4c70-8aa2-b31ad2e76813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:55.330840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad8c56-1e07-406a-9efb-0f2a54564571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:55.331933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad8c56-1e07-406a-9efb-0f2a54564571 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:55.335354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad8c56-1e07-406a-9efb-0f2a54564571 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:55.335956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad8c56-1e07-406a-9efb-0f2a54564571 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:55.337067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad8c56-1e07-406a-9efb-0f2a54564571 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:55.337593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad8c56-1e07-406a-9efb-0f2a54564571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:55.339336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad8c56-1e07-406a-9efb-0f2a54564571 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:55.340702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e6136fa4-84ff-4c70-8aa2-b31ad2e76813 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:55.344014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e6136fa4-84ff-4c70-8aa2-b31ad2e76813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:55.361161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 388/785] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:55.370212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad8c56-1e07-406a-9efb-0f2a54564571 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:55.382936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:55.477339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 398/786] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:41:55.503072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6136fa4-84ff-4c70-8aa2-b31ad2e76813 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:55.509065 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:55.542784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad8c56-1e07-406a-9efb-0f2a54564571 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:55.580037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4075633-74d1-476d-aa09-6333c1d229d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:55.581012 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4075633-74d1-476d-aa09-6333c1d229d3 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:55.581381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4075633-74d1-476d-aa09-6333c1d229d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:55.581668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4075633-74d1-476d-aa09-6333c1d229d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:55.595484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4075633-74d1-476d-aa09-6333c1d229d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:55.597190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4075633-74d1-476d-aa09-6333c1d229d3 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:55.612868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6136fa4-84ff-4c70-8aa2-b31ad2e76813 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-f4075633-74d1-476d-aa09-6333c1d229d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:55.616520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:55.617417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:55.618752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:55.618752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:55.618752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:55.618752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:55.618752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 389/787] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:55.634633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4075633-74d1-476d-aa09-6333c1d229d3 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:55.654049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f4c1f-eef7-470a-bbb4-bd2e82f5b3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:55.654869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f4c1f-eef7-470a-bbb4-bd2e82f5b3ee None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:55.655154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f4c1f-eef7-470a-bbb4-bd2e82f5b3ee None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:55.655430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f4c1f-eef7-470a-bbb4-bd2e82f5b3ee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:55.656085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f4c1f-eef7-470a-bbb4-bd2e82f5b3ee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:55.657482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f4c1f-eef7-470a-bbb4-bd2e82f5b3ee None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-c21f4c1f-eef7-470a-bbb4-bd2e82f5b3ee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:55.670789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:55.673104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:55.675231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 399/788] 10.222.0.17 () {56 vars in 1129 bytes} [Sun Jul 24 19:41:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:55.685396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f4c1f-eef7-470a-bbb4-bd2e82f5b3ee None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:55.700526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:55.757115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4075633-74d1-476d-aa09-6333c1d229d3 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:55.772726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:55.797237 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6196-199d-4b2f-b90f-06e1b2254b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:55.798354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6196-199d-4b2f-b90f-06e1b2254b2d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:55.798840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6196-199d-4b2f-b90f-06e1b2254b2d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:55.799317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6196-199d-4b2f-b90f-06e1b2254b2d None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:55.800243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6196-199d-4b2f-b90f-06e1b2254b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:55.800798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6196-199d-4b2f-b90f-06e1b2254b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:55.802529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6196-199d-4b2f-b90f-06e1b2254b2d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:55.816016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 390/789] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:55.822251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6196-199d-4b2f-b90f-06e1b2254b2d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:55.854821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f4c1f-eef7-470a-bbb4-bd2e82f5b3ee None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:55.948376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6196-199d-4b2f-b90f-06e1b2254b2d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:55.988563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48494f0-2458-4493-bea4-54970deb5e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:55.989802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48494f0-2458-4493-bea4-54970deb5e63 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:55.990301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48494f0-2458-4493-bea4-54970deb5e63 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:55.990775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48494f0-2458-4493-bea4-54970deb5e63 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:55.999679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48494f0-2458-4493-bea4-54970deb5e63 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:56.001538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48494f0-2458-4493-bea4-54970deb5e63 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-b48494f0-2458-4493-bea4-54970deb5e63 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:56.021737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:56.022779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:41:56.023715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:56.027396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 391/790] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:56.029692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5018-2066-40e6-8395-e55be465b4b0 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.030419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5018-2066-40e6-8395-e55be465b4b0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.030863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5018-2066-40e6-8395-e55be465b4b0 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.033181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 392/791] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:56.034058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee5018-2066-40e6-8395-e55be465b4b0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:56.045504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713a1b2-9491-4f9f-ad34-12c27eb90b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:56.046310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713a1b2-9491-4f9f-ad34-12c27eb90b58 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.046688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713a1b2-9491-4f9f-ad34-12c27eb90b58 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.047059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713a1b2-9491-4f9f-ad34-12c27eb90b58 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.047876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713a1b2-9491-4f9f-ad34-12c27eb90b58 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:56.048543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713a1b2-9491-4f9f-ad34-12c27eb90b58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:56.049913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713a1b2-9491-4f9f-ad34-12c27eb90b58 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:56.055292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 400/792] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Jul 24 19:41:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:56.056594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f60a0-3576-4a0e-a00a-d70e8bd312b7 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.057004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f60a0-3576-4a0e-a00a-d70e8bd312b7 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.057405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f60a0-3576-4a0e-a00a-d70e8bd312b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.075840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ec5f60a0-3576-4a0e-a00a-d70e8bd312b7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:56.077382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ec5f60a0-3576-4a0e-a00a-d70e8bd312b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:56.077745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:56.169183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 401/793] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:41:56.170311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f60a0-3576-4a0e-a00a-d70e8bd312b7 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:56.189203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee5018-2066-40e6-8395-e55be465b4b0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:56.199662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:56.246246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150d705-7cae-4ff1-8064-40afd31d8de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:56.247595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150d705-7cae-4ff1-8064-40afd31d8de3 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.248087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150d705-7cae-4ff1-8064-40afd31d8de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.248564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150d705-7cae-4ff1-8064-40afd31d8de3 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.250069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150d705-7cae-4ff1-8064-40afd31d8de3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:56.250608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150d705-7cae-4ff1-8064-40afd31d8de3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:56.252737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150d705-7cae-4ff1-8064-40afd31d8de3 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:56.271655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 393/794] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Jul 24 19:41:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:56.272903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150d705-7cae-4ff1-8064-40afd31d8de3 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:56.303431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f60a0-3576-4a0e-a00a-d70e8bd312b7 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:56.341632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cfc87-ace1-46ba-b0d3-3bca5a3e77c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:56.342309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cfc87-ace1-46ba-b0d3-3bca5a3e77c6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.342603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cfc87-ace1-46ba-b0d3-3bca5a3e77c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.342944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cfc87-ace1-46ba-b0d3-3bca5a3e77c6 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.343907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cfc87-ace1-46ba-b0d3-3bca5a3e77c6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:56.344107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cfc87-ace1-46ba-b0d3-3bca5a3e77c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:56.345577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cfc87-ace1-46ba-b0d3-3bca5a3e77c6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:56.351090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 402/795] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Jul 24 19:41:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:56.351907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cfc87-ace1-46ba-b0d3-3bca5a3e77c6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:56.370171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:56.380172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150d705-7cae-4ff1-8064-40afd31d8de3 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:56.417322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd924f-f697-4382-b561-a7e394df9aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:56.418452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd924f-f697-4382-b561-a7e394df9aee None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.418937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd924f-f697-4382-b561-a7e394df9aee None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.419457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd924f-f697-4382-b561-a7e394df9aee None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.431153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd924f-f697-4382-b561-a7e394df9aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-d8fd924f-f697-4382-b561-a7e394df9aee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:56.439210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:56.440131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:41:56.440982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:56.441767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:56.448217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 394/796] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:41:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:56.449284 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd924f-f697-4382-b561-a7e394df9aee None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:56.463828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cfc87-ace1-46ba-b0d3-3bca5a3e77c6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:56.502891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc005f6-b102-48b8-8bed-59d90aaa5800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:56.503725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc005f6-b102-48b8-8bed-59d90aaa5800 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.504332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc005f6-b102-48b8-8bed-59d90aaa5800 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.504453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc005f6-b102-48b8-8bed-59d90aaa5800 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.528163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc005f6-b102-48b8-8bed-59d90aaa5800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:56.528163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc005f6-b102-48b8-8bed-59d90aaa5800 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-ddc005f6-b102-48b8-8bed-59d90aaa5800 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:56.543338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:56.544462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:56.545434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:56.545434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:56.545434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:56.545434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:56.545434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 403/797] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:56.545819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc005f6-b102-48b8-8bed-59d90aaa5800 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:56.574452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd924f-f697-4382-b561-a7e394df9aee None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:56.610180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d477949-2a33-4abc-ab81-2808ae3c24f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:56.611089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d477949-2a33-4abc-ab81-2808ae3c24f9 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.611519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d477949-2a33-4abc-ab81-2808ae3c24f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.611936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d477949-2a33-4abc-ab81-2808ae3c24f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.612846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d477949-2a33-4abc-ab81-2808ae3c24f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:56.614377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d477949-2a33-4abc-ab81-2808ae3c24f9 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:56.673537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc005f6-b102-48b8-8bed-59d90aaa5800 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:56.724720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:56.734865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0e72e-bd87-4e29-be83-af9141df166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:56.736021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0e72e-bd87-4e29-be83-af9141df166e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.736522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0e72e-bd87-4e29-be83-af9141df166e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.737064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0e72e-bd87-4e29-be83-af9141df166e None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.742105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0e72e-bd87-4e29-be83-af9141df166e None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:56.747217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0e72e-bd87-4e29-be83-af9141df166e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:56.749128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0e72e-bd87-4e29-be83-af9141df166e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:56.757163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 395/798] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Jul 24 19:41:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:56.759338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d477949-2a33-4abc-ab81-2808ae3c24f9 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:56.761636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 404/799] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:41:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:56.772884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0e72e-bd87-4e29-be83-af9141df166e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:56.788580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:56.936783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d477949-2a33-4abc-ab81-2808ae3c24f9 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:56.978947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f97aed-513d-4723-b475-71245567ad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:56.980313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f97aed-513d-4723-b475-71245567ad35 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:56.980624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f97aed-513d-4723-b475-71245567ad35 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:56.980919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f97aed-513d-4723-b475-71245567ad35 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:56.981720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f97aed-513d-4723-b475-71245567ad35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:56.982037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f97aed-513d-4723-b475-71245567ad35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:56.984194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f97aed-513d-4723-b475-71245567ad35 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:56.999597 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 396/800] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:57.006073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f97aed-513d-4723-b475-71245567ad35 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:57.049000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0e72e-bd87-4e29-be83-af9141df166e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:57.098412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:57.108549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc5b93-0dc2-4ff9-b305-c27583b35090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:57.109438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc5b93-0dc2-4ff9-b305-c27583b35090 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.109722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc5b93-0dc2-4ff9-b305-c27583b35090 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.110047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc5b93-0dc2-4ff9-b305-c27583b35090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.124136 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc5b93-0dc2-4ff9-b305-c27583b35090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:57.125620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc5b93-0dc2-4ff9-b305-c27583b35090 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-46dc5b93-0dc2-4ff9-b305-c27583b35090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:57.149106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:57.150926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:57.161227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:57.161227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:57.161227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:57.161227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:57.161227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 405/801] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:57.161227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc5b93-0dc2-4ff9-b305-c27583b35090 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:57.177726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f97aed-513d-4723-b475-71245567ad35 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:57.217396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:57.247257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594b0de-6da7-4e22-a958-11ea7b2163f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:57.249085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594b0de-6da7-4e22-a958-11ea7b2163f3 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.249889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594b0de-6da7-4e22-a958-11ea7b2163f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.250701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594b0de-6da7-4e22-a958-11ea7b2163f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.271005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594b0de-6da7-4e22-a958-11ea7b2163f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:57.274208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594b0de-6da7-4e22-a958-11ea7b2163f3 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:57.286815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc5b93-0dc2-4ff9-b305-c27583b35090 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-8594b0de-6da7-4e22-a958-11ea7b2163f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:57.306648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:57.308309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:57.309933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:57.309933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:57.309933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:57.309933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:57.309933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:57.309933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:57.309933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:57.309933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:57.314290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 397/802] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:41:57.320486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594b0de-6da7-4e22-a958-11ea7b2163f3 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:57.331390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1d39b-7dfa-4ab5-ad66-c1397914cee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:57.332382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d39b-7dfa-4ab5-ad66-c1397914cee3 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.332905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d39b-7dfa-4ab5-ad66-c1397914cee3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.333398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d39b-7dfa-4ab5-ad66-c1397914cee3 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.334333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1d39b-7dfa-4ab5-ad66-c1397914cee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:57.335050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1d39b-7dfa-4ab5-ad66-c1397914cee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:57.336604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1d39b-7dfa-4ab5-ad66-c1397914cee3 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:57.349807 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 406/803] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:41:57.351499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4420-2216-4317-a6f5-bb63671a3941 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.351932 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4420-2216-4317-a6f5-bb63671a3941 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.352349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4420-2216-4317-a6f5-bb63671a3941 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.354658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 407/804] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:57.361538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee4420-2216-4317-a6f5-bb63671a3941 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:57.485362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee4420-2216-4317-a6f5-bb63671a3941 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:57.507862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:57.524793 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ee782-c2c4-4a10-9155-17d0436a61b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:57.525909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ee782-c2c4-4a10-9155-17d0436a61b9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.526398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ee782-c2c4-4a10-9155-17d0436a61b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.526857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ee782-c2c4-4a10-9155-17d0436a61b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.530246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594b0de-6da7-4e22-a958-11ea7b2163f3 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:57.535173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ee782-c2c4-4a10-9155-17d0436a61b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:57.536888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ee782-c2c4-4a10-9155-17d0436a61b9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-0f2ee782-c2c4-4a10-9155-17d0436a61b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:57.553183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:57.554128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:41:57.555069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:57.558132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:57.558550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 408/805] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:41:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:57.559935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdcf65-0df3-44d4-9a1f-4be25690dd0d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.560416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdcf65-0df3-44d4-9a1f-4be25690dd0d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.560921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdcf65-0df3-44d4-9a1f-4be25690dd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.577302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e382f7-6016-4a6c-8553-77fe59051219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:57.578024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e382f7-6016-4a6c-8553-77fe59051219 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.578396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e382f7-6016-4a6c-8553-77fe59051219 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.578674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e382f7-6016-4a6c-8553-77fe59051219 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.579528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e382f7-6016-4a6c-8553-77fe59051219 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:57.579733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-cbbdcf65-0df3-44d4-9a1f-4be25690dd0d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:57.579833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e382f7-6016-4a6c-8553-77fe59051219 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:57.581222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-cbbdcf65-0df3-44d4-9a1f-4be25690dd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:57.581331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e382f7-6016-4a6c-8553-77fe59051219 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:57.595944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 398/806] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:57.597214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e382f7-6016-4a6c-8553-77fe59051219 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:57.662253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 409/807] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:41:57.663405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdcf65-0df3-44d4-9a1f-4be25690dd0d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:57.736785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e382f7-6016-4a6c-8553-77fe59051219 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:57.775181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4585d-0672-4bc8-b4c0-a94e158a4a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:57.775949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4585d-0672-4bc8-b4c0-a94e158a4a0c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.776219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4585d-0672-4bc8-b4c0-a94e158a4a0c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.776490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4585d-0672-4bc8-b4c0-a94e158a4a0c None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.777409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4585d-0672-4bc8-b4c0-a94e158a4a0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:57.777717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4585d-0672-4bc8-b4c0-a94e158a4a0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:57.779144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4585d-0672-4bc8-b4c0-a94e158a4a0c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:57.787345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:57.796760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdcf65-0df3-44d4-9a1f-4be25690dd0d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:57.803834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 399/808] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Jul 24 19:41:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:57.804869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4585d-0672-4bc8-b4c0-a94e158a4a0c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:57.842642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d81c5-9dee-4e0d-baa0-a8eca51a8eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:57.843493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d81c5-9dee-4e0d-baa0-a8eca51a8eab None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.843781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d81c5-9dee-4e0d-baa0-a8eca51a8eab None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.844066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d81c5-9dee-4e0d-baa0-a8eca51a8eab None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/0849f31902da4613babb10a76c4548f3/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.846068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d81c5-9dee-4e0d-baa0-a8eca51a8eab None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=0849f31902da4613babb10a76c4548f3, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:57.847507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d81c5-9dee-4e0d-baa0-a8eca51a8eab None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:57.851786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:57.908883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4585d-0672-4bc8-b4c0-a94e158a4a0c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:57.923768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 410/809] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:41:57 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/0849f31902da4613babb10a76c4548f3/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:57.924386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d81c5-9dee-4e0d-baa0-a8eca51a8eab None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:57.979583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c62ac-d749-4398-939a-24b61364d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:57.980781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c62ac-d749-4398-939a-24b61364d827 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:57.981063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c62ac-d749-4398-939a-24b61364d827 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:57.981339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c62ac-d749-4398-939a-24b61364d827 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:57.997340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c62ac-d749-4398-939a-24b61364d827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-351c62ac-d749-4398-939a-24b61364d827 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:41:58.007595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:58.008610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:41:58.009533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:58.010862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 400/810] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:41:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:58.011046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763ddcd-4683-4afb-a4a4-58c581cffc79 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.011340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763ddcd-4683-4afb-a4a4-58c581cffc79 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.011617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763ddcd-4683-4afb-a4a4-58c581cffc79 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.012970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 401/811] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:58.015341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763ddcd-4683-4afb-a4a4-58c581cffc79 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:58.041829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d81c5-9dee-4e0d-baa0-a8eca51a8eab None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.088104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab9c6f-9772-4445-8aca-3f9c716c30b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.093296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9c6f-9772-4445-8aca-3f9c716c30b0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.093296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9c6f-9772-4445-8aca-3f9c716c30b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.093296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9c6f-9772-4445-8aca-3f9c716c30b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.106696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab9c6f-9772-4445-8aca-3f9c716c30b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:58.108182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab9c6f-9772-4445-8aca-3f9c716c30b0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-b2ab9c6f-9772-4445-8aca-3f9c716c30b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:58.129248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:58.134292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:58.135219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:58.135219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:58.135219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:58.135219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:58.135219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 411/812] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:58.135219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db1de3-f573-42dd-9466-5ad9fa4774bf None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.135219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db1de3-f573-42dd-9466-5ad9fa4774bf None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.135219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db1de3-f573-42dd-9466-5ad9fa4774bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.147109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:58.153888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a6db1de3-f573-42dd-9466-5ad9fa4774bf None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:58.155082 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a6db1de3-f573-42dd-9466-5ad9fa4774bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:58.166935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:58.173552 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763ddcd-4683-4afb-a4a4-58c581cffc79 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.213480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc099cf-929e-4ff5-9fcf-fdaa07a1a780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.214164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc099cf-929e-4ff5-9fcf-fdaa07a1a780 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.214538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc099cf-929e-4ff5-9fcf-fdaa07a1a780 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.214825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc099cf-929e-4ff5-9fcf-fdaa07a1a780 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.215651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc099cf-929e-4ff5-9fcf-fdaa07a1a780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:58.215971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc099cf-929e-4ff5-9fcf-fdaa07a1a780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:58.217480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc099cf-929e-4ff5-9fcf-fdaa07a1a780 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:58.226847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 402/813] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:58.228260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc099cf-929e-4ff5-9fcf-fdaa07a1a780 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:58.248737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 412/814] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:41:58.250997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db1de3-f573-42dd-9466-5ad9fa4774bf None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:58.383443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db1de3-f573-42dd-9466-5ad9fa4774bf None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.387519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc099cf-929e-4ff5-9fcf-fdaa07a1a780 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.433128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f670a-b49c-4584-9a81-374503a34886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.434009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570549b1-26d3-4072-8ba2-c32997377419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.434141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f670a-b49c-4584-9a81-374503a34886 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.434437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f670a-b49c-4584-9a81-374503a34886 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.434784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f670a-b49c-4584-9a81-374503a34886 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.434891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570549b1-26d3-4072-8ba2-c32997377419 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.435079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570549b1-26d3-4072-8ba2-c32997377419 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.435359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570549b1-26d3-4072-8ba2-c32997377419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.435808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f670a-b49c-4584-9a81-374503a34886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:58.436308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f670a-b49c-4584-9a81-374503a34886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:58.438123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f670a-b49c-4584-9a81-374503a34886 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:58.438746 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:58.448205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570549b1-26d3-4072-8ba2-c32997377419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:58.450346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570549b1-26d3-4072-8ba2-c32997377419 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:58.455682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 403/815] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:58.465200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f670a-b49c-4584-9a81-374503a34886 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-570549b1-26d3-4072-8ba2-c32997377419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:41:58.474286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:58.475190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:58.476080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:41:58.476080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:41:58.476080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:41:58.476080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:58.480717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 413/816] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:58.481826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570549b1-26d3-4072-8ba2-c32997377419 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:58.494852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:58.591776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f670a-b49c-4584-9a81-374503a34886 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.601667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570549b1-26d3-4072-8ba2-c32997377419 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.626129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169bfc0-b0db-4f46-967b-e053746dc11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.626926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169bfc0-b0db-4f46-967b-e053746dc11e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.627206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169bfc0-b0db-4f46-967b-e053746dc11e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.627482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169bfc0-b0db-4f46-967b-e053746dc11e None admin] PATH_INFO: `/v3/services/network` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.628139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169bfc0-b0db-4f46-967b-e053746dc11e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:58.629522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169bfc0-b0db-4f46-967b-e053746dc11e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:58.639934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1c953-acf1-46cb-8d9e-1314c70b3823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.641059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c953-acf1-46cb-8d9e-1314c70b3823 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-e169bfc0-b0db-4f46-967b-e053746dc11e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:58.641586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:58.642566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:41:58.643394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 404/817] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:58 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:58.644188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169bfc0-b0db-4f46-967b-e053746dc11e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:58.644549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c953-acf1-46cb-8d9e-1314c70b3823 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.645005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c953-acf1-46cb-8d9e-1314c70b3823 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.653522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1c953-acf1-46cb-8d9e-1314c70b3823 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:58.655261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1c953-acf1-46cb-8d9e-1314c70b3823 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-65e1c953-acf1-46cb-8d9e-1314c70b3823 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:41:58.670914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:58.672067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:58.673259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:41:58.673259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:41:58.673259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:41:58.673259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:41:58.673259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:41:58.673259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:41:58.673259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:41:58.673259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:58.675137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 414/818] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:58.676116 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1c953-acf1-46cb-8d9e-1314c70b3823 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:58.730328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:58.743265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:58.759473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169bfc0-b0db-4f46-967b-e053746dc11e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.785772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1c953-acf1-46cb-8d9e-1314c70b3823 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.795675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a213a69-50a6-4d56-a73c-f401375a9d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.796404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a213a69-50a6-4d56-a73c-f401375a9d8d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.796784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a213a69-50a6-4d56-a73c-f401375a9d8d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.797146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a213a69-50a6-4d56-a73c-f401375a9d8d None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.797942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a213a69-50a6-4d56-a73c-f401375a9d8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:58.798317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a213a69-50a6-4d56-a73c-f401375a9d8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:58.800151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a213a69-50a6-4d56-a73c-f401375a9d8d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:58.816331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 405/819] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:58.817531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a213a69-50a6-4d56-a73c-f401375a9d8d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:58.829183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18edd2-65bf-4080-aaf6-1a4ebc376ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.830268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18edd2-65bf-4080-aaf6-1a4ebc376ba7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.830767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18edd2-65bf-4080-aaf6-1a4ebc376ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.831275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18edd2-65bf-4080-aaf6-1a4ebc376ba7 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.832292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18edd2-65bf-4080-aaf6-1a4ebc376ba7 None admin] RBAC: Adding query filter params (name=network) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:58.832859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18edd2-65bf-4080-aaf6-1a4ebc376ba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:58.834646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18edd2-65bf-4080-aaf6-1a4ebc376ba7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:58.840331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 415/820] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:41:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:58.841487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18edd2-65bf-4080-aaf6-1a4ebc376ba7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:58.944693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a213a69-50a6-4d56-a73c-f401375a9d8d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.954699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18edd2-65bf-4080-aaf6-1a4ebc376ba7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:58.968372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:58.992149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45913a-b938-4675-9a35-60522533504f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.992846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45913a-b938-4675-9a35-60522533504f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.993704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45913a-b938-4675-9a35-60522533504f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:58.993993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45913a-b938-4675-9a35-60522533504f None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:58.994794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45913a-b938-4675-9a35-60522533504f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:58.995139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45913a-b938-4675-9a35-60522533504f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:58.996563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45913a-b938-4675-9a35-60522533504f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:58.997964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9126e-49ff-4f60-bc3f-96f4354e6d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:58.999287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9126e-49ff-4f60-bc3f-96f4354e6d3b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:58.999787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9126e-49ff-4f60-bc3f-96f4354e6d3b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:59.000791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9126e-49ff-4f60-bc3f-96f4354e6d3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:59.011295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9126e-49ff-4f60-bc3f-96f4354e6d3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:59.013084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9126e-49ff-4f60-bc3f-96f4354e6d3b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:59.016503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 406/821] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:59.017458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45913a-b938-4675-9a35-60522533504f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:59.027780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-37d9126e-49ff-4f60-bc3f-96f4354e6d3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:41:59.032064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:41:59.033017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:41:59.033958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:59.034495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 416/822] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Jul 24 19:41:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:41:59.035317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9126e-49ff-4f60-bc3f-96f4354e6d3b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:59.152400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9126e-49ff-4f60-bc3f-96f4354e6d3b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:59.153940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45913a-b938-4675-9a35-60522533504f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:59.197270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceebd8b-da9b-46b9-86bc-0d8f11c98d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:59.197869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceebd8b-da9b-46b9-86bc-0d8f11c98d11 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:59.198445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceebd8b-da9b-46b9-86bc-0d8f11c98d11 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:59.198445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceebd8b-da9b-46b9-86bc-0d8f11c98d11 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:59.199135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceebd8b-da9b-46b9-86bc-0d8f11c98d11 None admin] RBAC: Adding query filter params (type=network) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:59.199445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceebd8b-da9b-46b9-86bc-0d8f11c98d11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:59.199666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243aa80d-da20-408b-9bb1-5068a575c7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:59.200819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243aa80d-da20-408b-9bb1-5068a575c7e8 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:59.200928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceebd8b-da9b-46b9-86bc-0d8f11c98d11 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:59.201394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243aa80d-da20-408b-9bb1-5068a575c7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:59.201878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243aa80d-da20-408b-9bb1-5068a575c7e8 None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/8fb7202aa94640129f33a310f6d00876/roles/65e4ab8e291442e3858317b0aa6c2d8a` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:59.209250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 407/823] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:41:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:59.210761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceebd8b-da9b-46b9-86bc-0d8f11c98d11 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:59.223822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243aa80d-da20-408b-9bb1-5068a575c7e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=8fb7202aa94640129f33a310f6d00876, role_id=65e4ab8e291442e3858317b0aa6c2d8a)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:59.225362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243aa80d-da20-408b-9bb1-5068a575c7e8 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:59.287975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:59.300976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 417/824] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:41:59 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/8fb7202aa94640129f33a310f6d00876/roles/65e4ab8e291442e3858317b0aa6c2d8a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:41:59.302410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243aa80d-da20-408b-9bb1-5068a575c7e8 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:59.321508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceebd8b-da9b-46b9-86bc-0d8f11c98d11 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:59.347233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:59.366124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9fd91-4a7f-408c-a241-f77e459bda5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:59.367219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9fd91-4a7f-408c-a241-f77e459bda5f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:59.367710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9fd91-4a7f-408c-a241-f77e459bda5f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:59.368239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9fd91-4a7f-408c-a241-f77e459bda5f None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:59.369376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9fd91-4a7f-408c-a241-f77e459bda5f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:41:59.369906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9fd91-4a7f-408c-a241-f77e459bda5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:59.371607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9fd91-4a7f-408c-a241-f77e459bda5f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:59.392913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 408/825] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Jul 24 19:41:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:41:59.394759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e50b88-c650-4315-bc05-5b931b7e4712 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:59.395468 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e50b88-c650-4315-bc05-5b931b7e4712 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:59.395839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e50b88-c650-4315-bc05-5b931b7e4712 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:59.398062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 409/826] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:41:59.399768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e50b88-c650-4315-bc05-5b931b7e4712 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:59.469843 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243aa80d-da20-408b-9bb1-5068a575c7e8 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:59.515832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d6f7e-45d9-48d7-a35c-025ac283172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:59.516862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d6f7e-45d9-48d7-a35c-025ac283172f None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:59.517330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d6f7e-45d9-48d7-a35c-025ac283172f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:59.517784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d6f7e-45d9-48d7-a35c-025ac283172f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:59.518722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d6f7e-45d9-48d7-a35c-025ac283172f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:59.520279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d6f7e-45d9-48d7-a35c-025ac283172f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:59.544032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e50b88-c650-4315-bc05-5b931b7e4712 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:59.576197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:59.584054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41317489-6529-4163-bfee-9c6f4e193f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:59.584818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317489-6529-4163-bfee-9c6f4e193f80 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:59.585108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317489-6529-4163-bfee-9c6f4e193f80 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:59.585385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317489-6529-4163-bfee-9c6f4e193f80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:59.598671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 418/827] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Jul 24 19:41:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:41:59.600391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fa3e4-3fe2-43e3-b1c5-e9111525429b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:59.600896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fa3e4-3fe2-43e3-b1c5-e9111525429b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:59.601350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fa3e4-3fe2-43e3-b1c5-e9111525429b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:59.624731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e07fa3e4-3fe2-43e3-b1c5-e9111525429b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:41:59.626588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e07fa3e4-3fe2-43e3-b1c5-e9111525429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:41:59.710397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41317489-6529-4163-bfee-9c6f4e193f80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:59.712435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41317489-6529-4163-bfee-9c6f4e193f80 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:59.738128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 419/828] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:41:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:41:59.772899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fa3e4-3fe2-43e3-b1c5-e9111525429b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:59.776777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:59.888446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 410/829] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:41:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:41:59.893757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41317489-6529-4163-bfee-9c6f4e193f80 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:41:59.928449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fa3e4-3fe2-43e3-b1c5-e9111525429b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:41:59.934074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:41:59.980981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931e131-b5c9-47eb-b723-7ad335321503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:41:59.981953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e131-b5c9-47eb-b723-7ad335321503 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:41:59.982245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e131-b5c9-47eb-b723-7ad335321503 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:41:59.982520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e131-b5c9-47eb-b723-7ad335321503 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:41:59.983188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931e131-b5c9-47eb-b723-7ad335321503 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:41:59.984557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931e131-b5c9-47eb-b723-7ad335321503 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-d931e131-b5c9-47eb-b723-7ad335321503 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:41:59.999501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:00.000502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:00.001402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 420/830] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:41:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:00.007012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931e131-b5c9-47eb-b723-7ad335321503 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:00.029407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41317489-6529-4163-bfee-9c6f4e193f80 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:00.075264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0bfa3-a576-49a4-bb72-c274a7946286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:00.076519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0bfa3-a576-49a4-bb72-c274a7946286 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.076988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0bfa3-a576-49a4-bb72-c274a7946286 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.077440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0bfa3-a576-49a4-bb72-c274a7946286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.087347 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:00.090933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0bfa3-a576-49a4-bb72-c274a7946286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:00.092689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0bfa3-a576-49a4-bb72-c274a7946286 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-bda0bfa3-a576-49a4-bb72-c274a7946286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:00.114954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:00.116013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:00.116945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:00.116945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:00.116945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:00.116945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:00.117167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 411/831] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:00.123120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0bfa3-a576-49a4-bb72-c274a7946286 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:00.130791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931e131-b5c9-47eb-b723-7ad335321503 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:00.173463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e5694-1955-4ce2-99da-0281f5f79550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:00.174246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e5694-1955-4ce2-99da-0281f5f79550 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.174415 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e5694-1955-4ce2-99da-0281f5f79550 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.174712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e5694-1955-4ce2-99da-0281f5f79550 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.175928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e5694-1955-4ce2-99da-0281f5f79550 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:00.176242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e5694-1955-4ce2-99da-0281f5f79550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:00.177650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e5694-1955-4ce2-99da-0281f5f79550 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:00.183468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 421/832] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:42:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:00.184650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da39ca-f5b7-40c6-953a-9e6d89dc0199 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.185015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da39ca-f5b7-40c6-953a-9e6d89dc0199 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.185298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da39ca-f5b7-40c6-953a-9e6d89dc0199 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.188579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 422/833] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:00.189158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da39ca-f5b7-40c6-953a-9e6d89dc0199 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:00.222774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:00.242017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0bfa3-a576-49a4-bb72-c274a7946286 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:00.279109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104b4cc-9999-426c-aa7d-61e02cf01fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:00.279805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104b4cc-9999-426c-aa7d-61e02cf01fdb None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.280091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104b4cc-9999-426c-aa7d-61e02cf01fdb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.280370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104b4cc-9999-426c-aa7d-61e02cf01fdb None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.281233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104b4cc-9999-426c-aa7d-61e02cf01fdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:00.281543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104b4cc-9999-426c-aa7d-61e02cf01fdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:00.282909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104b4cc-9999-426c-aa7d-61e02cf01fdb None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:00.295349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 412/834] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:00.296207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da39ca-f5b7-40c6-953a-9e6d89dc0199 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:00.301090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf807a-f457-47c6-934e-3dec339279c1 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.301387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf807a-f457-47c6-934e-3dec339279c1 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.301669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf807a-f457-47c6-934e-3dec339279c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.318826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3fbf807a-f457-47c6-934e-3dec339279c1 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:00.320017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3fbf807a-f457-47c6-934e-3dec339279c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:00.325016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:00.329981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296d5aa-cd14-4571-80f9-aacaca1dd31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:00.330638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296d5aa-cd14-4571-80f9-aacaca1dd31e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.330925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296d5aa-cd14-4571-80f9-aacaca1dd31e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.331199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296d5aa-cd14-4571-80f9-aacaca1dd31e None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.331901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296d5aa-cd14-4571-80f9-aacaca1dd31e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:00.332207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296d5aa-cd14-4571-80f9-aacaca1dd31e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:00.333657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296d5aa-cd14-4571-80f9-aacaca1dd31e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:00.338799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 423/835] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:42:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:00.339577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296d5aa-cd14-4571-80f9-aacaca1dd31e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:00.411152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 413/836] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:00.437110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf807a-f457-47c6-934e-3dec339279c1 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:00.487719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296d5aa-cd14-4571-80f9-aacaca1dd31e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:00.533768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2514067-ca2c-443f-9704-0548c4e4e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:00.534499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2514067-ca2c-443f-9704-0548c4e4e041 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.534786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2514067-ca2c-443f-9704-0548c4e4e041 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.535056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2514067-ca2c-443f-9704-0548c4e4e041 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.545510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:00.548084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2514067-ca2c-443f-9704-0548c4e4e041 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:00.549819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2514067-ca2c-443f-9704-0548c4e4e041 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-c2514067-ca2c-443f-9704-0548c4e4e041 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:00.569213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:00.570358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:00.571994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:00.571994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:00.571994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:00.571994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:00.571994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:00.571994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:00.571994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:00.571994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:00.574248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 424/837] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:00.581958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2514067-ca2c-443f-9704-0548c4e4e041 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:00.592252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf807a-f457-47c6-934e-3dec339279c1 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:00.638286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41fdb9-c223-4432-83ee-4973edf0b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:00.639096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41fdb9-c223-4432-83ee-4973edf0b8cb None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.639380 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41fdb9-c223-4432-83ee-4973edf0b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.639656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41fdb9-c223-4432-83ee-4973edf0b8cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.653359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41fdb9-c223-4432-83ee-4973edf0b8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:00.662439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-ef41fdb9-c223-4432-83ee-4973edf0b8cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:00.662765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:00.663674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:00.664504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 414/838] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:42:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:00.670442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41fdb9-c223-4432-83ee-4973edf0b8cb None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:00.714129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2514067-ca2c-443f-9704-0548c4e4e041 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:00.751020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef72b14-f76a-4a59-942f-bd2a4191c82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:00.751630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef72b14-f76a-4a59-942f-bd2a4191c82b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.751916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef72b14-f76a-4a59-942f-bd2a4191c82b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.752187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef72b14-f76a-4a59-942f-bd2a4191c82b None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.753016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef72b14-f76a-4a59-942f-bd2a4191c82b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:00.753321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef72b14-f76a-4a59-942f-bd2a4191c82b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:00.754760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef72b14-f76a-4a59-942f-bd2a4191c82b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:00.774769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 425/839] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:00.777948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef72b14-f76a-4a59-942f-bd2a4191c82b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:00.806542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41fdb9-c223-4432-83ee-4973edf0b8cb None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:00.846292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:00.854192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3c201-038b-4acf-8466-77ef2345ca60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:00.855280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3c201-038b-4acf-8466-77ef2345ca60 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.855623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3c201-038b-4acf-8466-77ef2345ca60 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.860354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3c201-038b-4acf-8466-77ef2345ca60 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.860354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3c201-038b-4acf-8466-77ef2345ca60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:00.860354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3c201-038b-4acf-8466-77ef2345ca60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:00.860354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3c201-038b-4acf-8466-77ef2345ca60 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:00.872422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 415/840] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:42:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:00.882316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3c201-038b-4acf-8466-77ef2345ca60 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:00.905019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef72b14-f76a-4a59-942f-bd2a4191c82b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:00.922144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:00.943917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6b022-29a7-4314-9d7f-f0620585db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:00.945199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6b022-29a7-4314-9d7f-f0620585db80 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:00.945696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6b022-29a7-4314-9d7f-f0620585db80 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:00.946860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6b022-29a7-4314-9d7f-f0620585db80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:00.947877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6b022-29a7-4314-9d7f-f0620585db80 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=123b7f659c6a4ceba18ce268b1335542) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:00.948367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6b022-29a7-4314-9d7f-f0620585db80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:00.950025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6b022-29a7-4314-9d7f-f0620585db80 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:00.957247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 426/841] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:00 2022] GET /identity/v3/role_assignments?user.id=123b7f659c6a4ceba18ce268b1335542&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:01.017111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3c201-038b-4acf-8466-77ef2345ca60 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:01.058998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba989f-4a1e-48c2-bd67-4c518bb12461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:01.060130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba989f-4a1e-48c2-bd67-4c518bb12461 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:01.060590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba989f-4a1e-48c2-bd67-4c518bb12461 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:01.061090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba989f-4a1e-48c2-bd67-4c518bb12461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:01.073453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba989f-4a1e-48c2-bd67-4c518bb12461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:01.075292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba989f-4a1e-48c2-bd67-4c518bb12461 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-40ba989f-4a1e-48c2-bd67-4c518bb12461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:01.095862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:01.096952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:01.097829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:01.097829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:01.097829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:01.097829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:01.099995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 416/842] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:01.109776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6b022-29a7-4314-9d7f-f0620585db80 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:01.222720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6b022-29a7-4314-9d7f-f0620585db80 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:01.260035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc8a11-8a8b-41e3-9c3e-f860906cffa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:01.260848 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc8a11-8a8b-41e3-9c3e-f860906cffa8 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:01.261138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc8a11-8a8b-41e3-9c3e-f860906cffa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:01.261423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc8a11-8a8b-41e3-9c3e-f860906cffa8 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:01.262155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc8a11-8a8b-41e3-9c3e-f860906cffa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:01.262470 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc8a11-8a8b-41e3-9c3e-f860906cffa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:01.263849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc8a11-8a8b-41e3-9c3e-f860906cffa8 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:01.277791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 427/843] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:01.283148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba989f-4a1e-48c2-bd67-4c518bb12461 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:01.336055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:01.392141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba989f-4a1e-48c2-bd67-4c518bb12461 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:01.430114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e95e00-ea03-4782-8a7d-8c3c72f53b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:01.431583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e95e00-ea03-4782-8a7d-8c3c72f53b0c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:01.431925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e95e00-ea03-4782-8a7d-8c3c72f53b0c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:01.432152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e95e00-ea03-4782-8a7d-8c3c72f53b0c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:01.440904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e95e00-ea03-4782-8a7d-8c3c72f53b0c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:01.442395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e95e00-ea03-4782-8a7d-8c3c72f53b0c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-80e95e00-ea03-4782-8a7d-8c3c72f53b0c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:42:01.458559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:01.459520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:01.460589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 417/844] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:42:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:01.467502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc8a11-8a8b-41e3-9c3e-f860906cffa8 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:01.515212 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:01.589934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc8a11-8a8b-41e3-9c3e-f860906cffa8 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:01.627589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4a8a7-0c2e-4547-a902-94100fae223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:01.628280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a8a7-0c2e-4547-a902-94100fae223f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:01.628569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a8a7-0c2e-4547-a902-94100fae223f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:01.628879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a8a7-0c2e-4547-a902-94100fae223f None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:01.629087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76cf8a-af15-48e5-bbbd-bdc83aa06fbe None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:01.629377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76cf8a-af15-48e5-bbbd-bdc83aa06fbe None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:01.629784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76cf8a-af15-48e5-bbbd-bdc83aa06fbe None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:01.629937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4a8a7-0c2e-4547-a902-94100fae223f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:01.630363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4a8a7-0c2e-4547-a902-94100fae223f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:01.631580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 418/845] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:42:01.632744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4a8a7-0c2e-4547-a902-94100fae223f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:01.637705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbeb5e-200c-4208-9665-8c6b92e30230 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:01.638260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbeb5e-200c-4208-9665-8c6b92e30230 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:01.638764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbeb5e-200c-4208-9665-8c6b92e30230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:01.653234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 428/846] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Jul 24 19:42:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:01.656255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-adcbeb5e-200c-4208-9665-8c6b92e30230 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:01.660689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-adcbeb5e-200c-4208-9665-8c6b92e30230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:01.661644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4a8a7-0c2e-4547-a902-94100fae223f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:01.740676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 419/847] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:01.758059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbeb5e-200c-4208-9665-8c6b92e30230 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:01.781816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:01.786387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4a8a7-0c2e-4547-a902-94100fae223f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:01.798326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:01.833420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7beeadc-e004-48c4-87d6-acbf65baeda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:01.834179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7beeadc-e004-48c4-87d6-acbf65baeda3 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:01.834737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7beeadc-e004-48c4-87d6-acbf65baeda3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:01.835426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7beeadc-e004-48c4-87d6-acbf65baeda3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:01.877723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbeb5e-200c-4208-9665-8c6b92e30230 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:01.926451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7beeadc-e004-48c4-87d6-acbf65baeda3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:01.927364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078022b-dfab-43b8-918c-d5086aa49030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:01.928417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078022b-dfab-43b8-918c-d5086aa49030 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:01.928803 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078022b-dfab-43b8-918c-d5086aa49030 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:01.929161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078022b-dfab-43b8-918c-d5086aa49030 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:01.929328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7beeadc-e004-48c4-87d6-acbf65baeda3 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:01.946252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078022b-dfab-43b8-918c-d5086aa49030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-7078022b-dfab-43b8-918c-d5086aa49030 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:01.955262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:01.956258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:42:01.958934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:01.968931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 420/848] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:42:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:01.970217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078022b-dfab-43b8-918c-d5086aa49030 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:02.063847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:02.072744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 429/849] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:42:02.075263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7beeadc-e004-48c4-87d6-acbf65baeda3 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:02.096585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:02.106141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078022b-dfab-43b8-918c-d5086aa49030 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:02.151549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb5ff5-fdb1-495f-a017-5751019d72a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:02.152608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb5ff5-fdb1-495f-a017-5751019d72a4 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:02.153012 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb5ff5-fdb1-495f-a017-5751019d72a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:02.153387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb5ff5-fdb1-495f-a017-5751019d72a4 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:02.154502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb5ff5-fdb1-495f-a017-5751019d72a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:02.155054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb5ff5-fdb1-495f-a017-5751019d72a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:02.157071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb5ff5-fdb1-495f-a017-5751019d72a4 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:02.169188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 421/850] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:42:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:02.175717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb5ff5-fdb1-495f-a017-5751019d72a4 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:02.211248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7beeadc-e004-48c4-87d6-acbf65baeda3 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:02.255597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a70ef-5d21-4f19-afb6-38f94f5cb102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:02.257313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a70ef-5d21-4f19-afb6-38f94f5cb102 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:02.257891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a70ef-5d21-4f19-afb6-38f94f5cb102 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:02.258484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a70ef-5d21-4f19-afb6-38f94f5cb102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:02.272482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a70ef-5d21-4f19-afb6-38f94f5cb102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:02.274710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a70ef-5d21-4f19-afb6-38f94f5cb102 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-475a70ef-5d21-4f19-afb6-38f94f5cb102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:02.298764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:02.299912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:02.301056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:02.301056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:02.301056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:02.301056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:02.304742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 430/851] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:02.309282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a70ef-5d21-4f19-afb6-38f94f5cb102 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:02.322842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb5ff5-fdb1-495f-a017-5751019d72a4 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:02.362515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f46236-2aa0-45fc-856a-0411a7252f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:02.363627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f46236-2aa0-45fc-856a-0411a7252f67 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:02.364226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f46236-2aa0-45fc-856a-0411a7252f67 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:02.364730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f46236-2aa0-45fc-856a-0411a7252f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:02.373500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:02.378491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f46236-2aa0-45fc-856a-0411a7252f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:02.380259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f46236-2aa0-45fc-856a-0411a7252f67 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-81f46236-2aa0-45fc-856a-0411a7252f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:02.399735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:02.400760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:02.402051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:02.402051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:02.402051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:02.402051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:02.412895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 422/852] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:02.416125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f46236-2aa0-45fc-856a-0411a7252f67 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:02.432756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:02.434110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a70ef-5d21-4f19-afb6-38f94f5cb102 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:02.482494 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50548784-6a32-43ef-b467-728b0507d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:02.483315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50548784-6a32-43ef-b467-728b0507d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:02.483700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50548784-6a32-43ef-b467-728b0507d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:02.484355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50548784-6a32-43ef-b467-728b0507d5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:02.485227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50548784-6a32-43ef-b467-728b0507d5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:02.485635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50548784-6a32-43ef-b467-728b0507d5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:02.487314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50548784-6a32-43ef-b467-728b0507d5c5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:02.509074 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 431/853] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:02.513269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50548784-6a32-43ef-b467-728b0507d5c5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:02.553346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f46236-2aa0-45fc-856a-0411a7252f67 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:02.605373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae275d45-0587-4269-b816-4b7c0e6d4590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:02.606330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae275d45-0587-4269-b816-4b7c0e6d4590 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:02.606825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae275d45-0587-4269-b816-4b7c0e6d4590 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:02.607283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae275d45-0587-4269-b816-4b7c0e6d4590 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:02.608359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae275d45-0587-4269-b816-4b7c0e6d4590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:02.608908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae275d45-0587-4269-b816-4b7c0e6d4590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:02.610467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae275d45-0587-4269-b816-4b7c0e6d4590 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:02.623844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 423/854] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:02.625173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68459d53-0ec1-41b1-b1a7-207dc96f7f7b None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:02.625561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68459d53-0ec1-41b1-b1a7-207dc96f7f7b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:02.625841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68459d53-0ec1-41b1-b1a7-207dc96f7f7b None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:02.627244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 424/855] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:02.632210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68459d53-0ec1-41b1-b1a7-207dc96f7f7b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:02.646201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50548784-6a32-43ef-b467-728b0507d5c5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:02.662599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:02.691763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec5478-b4e8-4acd-8e23-61f6b200e3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:02.693129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec5478-b4e8-4acd-8e23-61f6b200e3ab None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:02.693569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec5478-b4e8-4acd-8e23-61f6b200e3ab None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:02.693970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec5478-b4e8-4acd-8e23-61f6b200e3ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:02.703855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec5478-b4e8-4acd-8e23-61f6b200e3ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:02.705447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec5478-b4e8-4acd-8e23-61f6b200e3ab None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a9ec5478-b4e8-4acd-8e23-61f6b200e3ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:02.720203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:02.721185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:02.722141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:02.722141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:02.722141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:02.722141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:02.722141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:02.722141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:02.722141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:02.722141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:02.722141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 432/856] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:02.722804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1977d1-103d-417b-85ee-db47254f4738 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:02.723744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1977d1-103d-417b-85ee-db47254f4738 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:02.723744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1977d1-103d-417b-85ee-db47254f4738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:02.735754 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:02.741096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-3a1977d1-103d-417b-85ee-db47254f4738 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:02.742296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3a1977d1-103d-417b-85ee-db47254f4738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:02.782848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68459d53-0ec1-41b1-b1a7-207dc96f7f7b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:02.823731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 433/857] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:02.824984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1977d1-103d-417b-85ee-db47254f4738 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:02.835043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8e187-66a4-4a14-a5bd-2e4b7a4a38ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:02.835878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8e187-66a4-4a14-a5bd-2e4b7a4a38ad None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:02.836164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8e187-66a4-4a14-a5bd-2e4b7a4a38ad None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:02.836442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8e187-66a4-4a14-a5bd-2e4b7a4a38ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:02.846390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8e187-66a4-4a14-a5bd-2e4b7a4a38ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:02.847843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8e187-66a4-4a14-a5bd-2e4b7a4a38ad None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-1ab8e187-66a4-4a14-a5bd-2e4b7a4a38ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:42:02.864230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:02.865330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:02.866293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 425/858] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:42:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:02.866887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8e187-66a4-4a14-a5bd-2e4b7a4a38ad None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:02.962825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:02.965986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1977d1-103d-417b-85ee-db47254f4738 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:02.994281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8e187-66a4-4a14-a5bd-2e4b7a4a38ad None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:03.013710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81022b-0cdb-42ca-ac17-9d0db8ad8556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:03.014377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81022b-0cdb-42ca-ac17-9d0db8ad8556 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:03.014663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81022b-0cdb-42ca-ac17-9d0db8ad8556 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:03.014940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81022b-0cdb-42ca-ac17-9d0db8ad8556 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:03.015730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81022b-0cdb-42ca-ac17-9d0db8ad8556 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:03.016036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81022b-0cdb-42ca-ac17-9d0db8ad8556 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:03.017501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81022b-0cdb-42ca-ac17-9d0db8ad8556 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:03.025359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:03.030876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 434/859] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:03.032019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81022b-0cdb-42ca-ac17-9d0db8ad8556 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:03.039230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b4b62-7063-4e9b-b17d-7a6baf469d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:03.040687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b4b62-7063-4e9b-b17d-7a6baf469d1f None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:03.041078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b4b62-7063-4e9b-b17d-7a6baf469d1f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:03.041707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b4b62-7063-4e9b-b17d-7a6baf469d1f None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:03.042543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b4b62-7063-4e9b-b17d-7a6baf469d1f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:03.044039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b4b62-7063-4e9b-b17d-7a6baf469d1f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:03.124735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 426/860] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:42:02 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:03.124735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b4b62-7063-4e9b-b17d-7a6baf469d1f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:03.153440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81022b-0cdb-42ca-ac17-9d0db8ad8556 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:03.217391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf0437-9b7e-4e1f-a118-c0ed1b1ce66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:03.219029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf0437-9b7e-4e1f-a118-c0ed1b1ce66a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:03.219524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf0437-9b7e-4e1f-a118-c0ed1b1ce66a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:03.220256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf0437-9b7e-4e1f-a118-c0ed1b1ce66a None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:03.221348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf0437-9b7e-4e1f-a118-c0ed1b1ce66a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:03.222011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf0437-9b7e-4e1f-a118-c0ed1b1ce66a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:03.224584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf0437-9b7e-4e1f-a118-c0ed1b1ce66a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:03.249665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 435/861] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Jul 24 19:42:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:03.251468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a4fa2-6e74-47d3-946d-edea74b80eb2 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:03.251952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a4fa2-6e74-47d3-946d-edea74b80eb2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:03.252378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a4fa2-6e74-47d3-946d-edea74b80eb2 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:03.254309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 436/862] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:03.260218 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a4fa2-6e74-47d3-946d-edea74b80eb2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:03.265554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b4b62-7063-4e9b-b17d-7a6baf469d1f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:03.279066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:03.326271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d573d-0b16-4364-9058-6610d9907cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:03.326818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d573d-0b16-4364-9058-6610d9907cf7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:03.327105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d573d-0b16-4364-9058-6610d9907cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:03.327386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d573d-0b16-4364-9058-6610d9907cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:03.328258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d573d-0b16-4364-9058-6610d9907cf7 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=0849f31902da4613babb10a76c4548f3) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:03.328574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d573d-0b16-4364-9058-6610d9907cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:03.330055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d573d-0b16-4364-9058-6610d9907cf7 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:03.336219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 427/863] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:03 2022] GET /identity/v3/role_assignments?user.id=0849f31902da4613babb10a76c4548f3&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:03.338066 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c100-b237-4aa1-8eaf-d11041c7d01b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:03.338357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c100-b237-4aa1-8eaf-d11041c7d01b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:03.338636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c100-b237-4aa1-8eaf-d11041c7d01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:03.398366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a4fa2-6e74-47d3-946d-edea74b80eb2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:03.442294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78d42d-2431-4b3d-a9d3-6605d25e04df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:03.442996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78d42d-2431-4b3d-a9d3-6605d25e04df None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:03.443284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78d42d-2431-4b3d-a9d3-6605d25e04df None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:03.443564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78d42d-2431-4b3d-a9d3-6605d25e04df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:03.499167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-5105c100-b237-4aa1-8eaf-d11041c7d01b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:03.500411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5105c100-b237-4aa1-8eaf-d11041c7d01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:03.529938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78d42d-2431-4b3d-a9d3-6605d25e04df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:03.531449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78d42d-2431-4b3d-a9d3-6605d25e04df None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:03.558013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:03.590409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:03.625640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 428/864] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:03.650561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105c100-b237-4aa1-8eaf-d11041c7d01b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:03.668407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 437/865] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:03 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:42:03.677714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78d42d-2431-4b3d-a9d3-6605d25e04df None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:03.785937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105c100-b237-4aa1-8eaf-d11041c7d01b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:03.799747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78d42d-2431-4b3d-a9d3-6605d25e04df None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:03.819163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:03.829735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acfeec-9723-4e35-a272-8414dbc3ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:03.830689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acfeec-9723-4e35-a272-8414dbc3ef55 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:03.830976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acfeec-9723-4e35-a272-8414dbc3ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:03.831252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acfeec-9723-4e35-a272-8414dbc3ef55 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:03.832003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acfeec-9723-4e35-a272-8414dbc3ef55 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:03.833337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acfeec-9723-4e35-a272-8414dbc3ef55 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:03.845476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a64fd-934f-4da1-b303-8c3cbce3e087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:03.846913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a64fd-934f-4da1-b303-8c3cbce3e087 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:03.847803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a64fd-934f-4da1-b303-8c3cbce3e087 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:03.848288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a64fd-934f-4da1-b303-8c3cbce3e087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-84acfeec-9723-4e35-a272-8414dbc3ef55 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:03.851365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:03.852925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 24 19:42:03.853764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:03.857018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 429/866] 10.222.0.17 () {56 vars in 1132 bytes} [Sun Jul 24 19:42:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:03.859537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acfeec-9723-4e35-a272-8414dbc3ef55 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:03.866556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a64fd-934f-4da1-b303-8c3cbce3e087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:03.868384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a64fd-934f-4da1-b303-8c3cbce3e087 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-ed5a64fd-934f-4da1-b303-8c3cbce3e087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:03.891420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:03.892368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:03.893276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:03.893276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:03.893276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:03.893276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:03.899571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 438/867] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:03.901275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a64fd-934f-4da1-b303-8c3cbce3e087 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:03.922160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:03.987722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acfeec-9723-4e35-a272-8414dbc3ef55 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:04.027399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f15d1-99ac-4f33-aeeb-bbae7ff573fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:04.028457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f15d1-99ac-4f33-aeeb-bbae7ff573fd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:04.028973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f15d1-99ac-4f33-aeeb-bbae7ff573fd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:04.029489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f15d1-99ac-4f33-aeeb-bbae7ff573fd None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:04.030391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f15d1-99ac-4f33-aeeb-bbae7ff573fd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:04.030912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f15d1-99ac-4f33-aeeb-bbae7ff573fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:04.032479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f15d1-99ac-4f33-aeeb-bbae7ff573fd None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:04.042819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a64fd-934f-4da1-b303-8c3cbce3e087 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:04.043442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 430/868] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:04.044542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f15d1-99ac-4f33-aeeb-bbae7ff573fd None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:04.095645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126324b-020f-4ed2-aaa3-aff454081230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:04.096772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126324b-020f-4ed2-aaa3-aff454081230 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:04.097303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126324b-020f-4ed2-aaa3-aff454081230 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:04.097878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126324b-020f-4ed2-aaa3-aff454081230 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:04.098969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126324b-020f-4ed2-aaa3-aff454081230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:04.099502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126324b-020f-4ed2-aaa3-aff454081230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:04.101154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126324b-020f-4ed2-aaa3-aff454081230 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:04.114757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 439/869] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:04.122034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126324b-020f-4ed2-aaa3-aff454081230 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:04.160334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:04.197816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f15d1-99ac-4f33-aeeb-bbae7ff573fd None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:04.217409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:04.253340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126324b-020f-4ed2-aaa3-aff454081230 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:04.255801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb692ba-d704-43e8-a8be-ad78a84af916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:04.257330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb692ba-d704-43e8-a8be-ad78a84af916 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:04.258090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb692ba-d704-43e8-a8be-ad78a84af916 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:04.258885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb692ba-d704-43e8-a8be-ad78a84af916 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:04.260499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb692ba-d704-43e8-a8be-ad78a84af916 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:04.261551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb692ba-d704-43e8-a8be-ad78a84af916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:04.264791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb692ba-d704-43e8-a8be-ad78a84af916 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:04.280664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 431/870] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:04 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:04.292385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd4ffb-d610-49d5-a1b7-10c2d66d3d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:04.293315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd4ffb-d610-49d5-a1b7-10c2d66d3d23 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:04.293641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd4ffb-d610-49d5-a1b7-10c2d66d3d23 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:04.293795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd4ffb-d610-49d5-a1b7-10c2d66d3d23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:04.302775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd4ffb-d610-49d5-a1b7-10c2d66d3d23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:04.304246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd4ffb-d610-49d5-a1b7-10c2d66d3d23 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-96cd4ffb-d610-49d5-a1b7-10c2d66d3d23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:04.320050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:04.321823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:04.323669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:04.323669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:04.323669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:04.323669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:04.323669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:04.323669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:04.323669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:04.323669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:04.323669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 440/871] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:04.335086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb692ba-d704-43e8-a8be-ad78a84af916 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:04.494767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb692ba-d704-43e8-a8be-ad78a84af916 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:04.533830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5aaec-81ba-4028-8956-95aedabf9f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:04.534911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5aaec-81ba-4028-8956-95aedabf9f51 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:04.535407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5aaec-81ba-4028-8956-95aedabf9f51 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:04.535921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5aaec-81ba-4028-8956-95aedabf9f51 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:04.536948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5aaec-81ba-4028-8956-95aedabf9f51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:04.537457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5aaec-81ba-4028-8956-95aedabf9f51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:04.539168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5aaec-81ba-4028-8956-95aedabf9f51 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:04.554107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 432/872] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:04.561083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd4ffb-d610-49d5-a1b7-10c2d66d3d23 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:04.656849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:04.673778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd4ffb-d610-49d5-a1b7-10c2d66d3d23 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:04.713569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7009e24-81fb-4e0a-9b9d-5258cea4a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:04.714249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7009e24-81fb-4e0a-9b9d-5258cea4a0c6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:04.714539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7009e24-81fb-4e0a-9b9d-5258cea4a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:04.714821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7009e24-81fb-4e0a-9b9d-5258cea4a0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:04.715694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7009e24-81fb-4e0a-9b9d-5258cea4a0c6 None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=8fb7202aa94640129f33a310f6d00876) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:04.716013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7009e24-81fb-4e0a-9b9d-5258cea4a0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:04.717542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7009e24-81fb-4e0a-9b9d-5258cea4a0c6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:04.724783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 441/873] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:04 2022] GET /identity/v3/role_assignments?user.id=8fb7202aa94640129f33a310f6d00876&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:05.346499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5652b-1b83-4b63-97e8-0ef0f486e629 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:05.347028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5652b-1b83-4b63-97e8-0ef0f486e629 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:05.347106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5652b-1b83-4b63-97e8-0ef0f486e629 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:05.348824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 433/874] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:05.355360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6aa3c-6f20-4e1a-8363-72a689bc9203 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:05.355844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6aa3c-6f20-4e1a-8363-72a689bc9203 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:05.355920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6aa3c-6f20-4e1a-8363-72a689bc9203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:05.374372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-82a6aa3c-6f20-4e1a-8363-72a689bc9203 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:05.375583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-82a6aa3c-6f20-4e1a-8363-72a689bc9203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:05.456796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 442/875] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:05.474131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5652b-1b83-4b63-97e8-0ef0f486e629 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:05.514881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:05.523616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9f019-fe07-4a20-a1aa-d9c656ba61b3 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:05.523998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9f019-fe07-4a20-a1aa-d9c656ba61b3 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:05.524212 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9f019-fe07-4a20-a1aa-d9c656ba61b3 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:05.525957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 443/876] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:05.531535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860dce4-afbc-49dd-99eb-85be874d29da None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:05.544185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860dce4-afbc-49dd-99eb-85be874d29da None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:05.544185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860dce4-afbc-49dd-99eb-85be874d29da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:05.550419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d860dce4-afbc-49dd-99eb-85be874d29da None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:05.551769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d860dce4-afbc-49dd-99eb-85be874d29da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:05.605938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5652b-1b83-4b63-97e8-0ef0f486e629 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:05.632929 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 444/877] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:05.647135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca275da-201d-488c-bbfa-ea81b5840135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:05.647956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca275da-201d-488c-bbfa-ea81b5840135 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:05.648247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca275da-201d-488c-bbfa-ea81b5840135 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:05.648532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca275da-201d-488c-bbfa-ea81b5840135 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:05.649278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca275da-201d-488c-bbfa-ea81b5840135 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:05.650678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca275da-201d-488c-bbfa-ea81b5840135 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:05.653041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860dce4-afbc-49dd-99eb-85be874d29da None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:05.668552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-cca275da-201d-488c-bbfa-ea81b5840135 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:05.670304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:05.671392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 24 19:42:05.672212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:05.672994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 434/878] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:42:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:05.678578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca275da-201d-488c-bbfa-ea81b5840135 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:05.766283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860dce4-afbc-49dd-99eb-85be874d29da None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:05.802791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f3336-a86f-4893-8713-1cda37de82ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:05.803714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f3336-a86f-4893-8713-1cda37de82ec None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:05.804065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f3336-a86f-4893-8713-1cda37de82ec None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:05.804420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f3336-a86f-4893-8713-1cda37de82ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:05.805211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f3336-a86f-4893-8713-1cda37de82ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:05.806725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f3336-a86f-4893-8713-1cda37de82ec None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:05.810734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:05.815247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca275da-201d-488c-bbfa-ea81b5840135 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-e88f3336-a86f-4893-8713-1cda37de82ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:05.819247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:05.827147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:05.828024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 445/879] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:05.828854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f3336-a86f-4893-8713-1cda37de82ec None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:05.860525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0019f51-5481-4f8e-ae44-44e912fa9d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:05.861996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0019f51-5481-4f8e-ae44-44e912fa9d32 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:05.862539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0019f51-5481-4f8e-ae44-44e912fa9d32 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:05.862995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0019f51-5481-4f8e-ae44-44e912fa9d32 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:05.864493 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0019f51-5481-4f8e-ae44-44e912fa9d32 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:05.864985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0019f51-5481-4f8e-ae44-44e912fa9d32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:05.866772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0019f51-5481-4f8e-ae44-44e912fa9d32 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:05.879015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 435/880] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:42:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:05.880788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0019f51-5481-4f8e-ae44-44e912fa9d32 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:05.918839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:05.945513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f3336-a86f-4893-8713-1cda37de82ec None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:05.984479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ac429-4c7f-4568-9235-7ccb719b985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:05.985235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ac429-4c7f-4568-9235-7ccb719b985a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:05.985524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ac429-4c7f-4568-9235-7ccb719b985a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:05.985805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ac429-4c7f-4568-9235-7ccb719b985a None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:05.986522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ac429-4c7f-4568-9235-7ccb719b985a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:05.986916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ac429-4c7f-4568-9235-7ccb719b985a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:05.988311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ac429-4c7f-4568-9235-7ccb719b985a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:05.999206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 446/881] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:06.000617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ac429-4c7f-4568-9235-7ccb719b985a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:06.006489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0019f51-5481-4f8e-ae44-44e912fa9d32 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:06.044010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232fec3-7b59-404c-8e6d-90584931aaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:06.045112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232fec3-7b59-404c-8e6d-90584931aaa1 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:06.045631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232fec3-7b59-404c-8e6d-90584931aaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:06.046137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232fec3-7b59-404c-8e6d-90584931aaa1 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:06.047064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232fec3-7b59-404c-8e6d-90584931aaa1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:06.047606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232fec3-7b59-404c-8e6d-90584931aaa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:06.049239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232fec3-7b59-404c-8e6d-90584931aaa1 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:06.060697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 436/882] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:42:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:06.062329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232fec3-7b59-404c-8e6d-90584931aaa1 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:06.092587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:06.111780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ac429-4c7f-4568-9235-7ccb719b985a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:06.149366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a5807-3c5f-4f2a-a811-16ff11b77bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:06.160460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a5807-3c5f-4f2a-a811-16ff11b77bf4 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:06.160460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a5807-3c5f-4f2a-a811-16ff11b77bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:06.160460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a5807-3c5f-4f2a-a811-16ff11b77bf4 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:06.160460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a5807-3c5f-4f2a-a811-16ff11b77bf4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:06.160460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a5807-3c5f-4f2a-a811-16ff11b77bf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:06.160460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a5807-3c5f-4f2a-a811-16ff11b77bf4 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:06.164789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 447/883] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:06.176211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232fec3-7b59-404c-8e6d-90584931aaa1 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:06.218691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe8cb-dde1-4c8a-a523-e46b887bf2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:06.219839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe8cb-dde1-4c8a-a523-e46b887bf2a5 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:06.220377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe8cb-dde1-4c8a-a523-e46b887bf2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:06.220931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe8cb-dde1-4c8a-a523-e46b887bf2a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:06.222095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe8cb-dde1-4c8a-a523-e46b887bf2a5 None admin] RBAC: Adding query filter params (interface=public, service_id=037feb8fd2b348e6a1013a97f29cef7d, region_id=RegionOne) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:06.222637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe8cb-dde1-4c8a-a523-e46b887bf2a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:06.224330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe8cb-dde1-4c8a-a523-e46b887bf2a5 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:06.239297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 437/884] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Jul 24 19:42:06 2022] GET /identity/v3/endpoints?service_id=037feb8fd2b348e6a1013a97f29cef7d&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:06.241235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a5807-3c5f-4f2a-a811-16ff11b77bf4 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:06.244913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:06.363726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a5807-3c5f-4f2a-a811-16ff11b77bf4 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:06.401772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ed8ff-ee88-4c1f-847f-4c0744e70ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:06.402466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ed8ff-ee88-4c1f-847f-4c0744e70ca5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:06.402750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ed8ff-ee88-4c1f-847f-4c0744e70ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:06.403029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ed8ff-ee88-4c1f-847f-4c0744e70ca5 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:06.404178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ed8ff-ee88-4c1f-847f-4c0744e70ca5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:06.405611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ed8ff-ee88-4c1f-847f-4c0744e70ca5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:06.412239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 448/885] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Jul 24 19:42:06 2022] GET /identity/v3/services => generated 1011 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 19:42:06.481579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbea7f3-c1f7-4f58-a981-370c299a8699 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:06.483058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbea7f3-c1f7-4f58-a981-370c299a8699 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:06.483796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbea7f3-c1f7-4f58-a981-370c299a8699 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:06.487266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 438/886] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:06.497552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eaf66-ee98-4e9d-85be-4d93c8ed504a None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:06.498515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eaf66-ee98-4e9d-85be-4d93c8ed504a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:06.499271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eaf66-ee98-4e9d-85be-4d93c8ed504a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:06.528768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-164eaf66-ee98-4e9d-85be-4d93c8ed504a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:06.531294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-164eaf66-ee98-4e9d-85be-4d93c8ed504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:06.629329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:06.638763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 449/887] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:06 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:06.661907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbea7f3-c1f7-4f58-a981-370c299a8699 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:06.776423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:06.787663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbea7f3-c1f7-4f58-a981-370c299a8699 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:06.826485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60102a1d-283c-457d-b298-d158475cdebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:06.827186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60102a1d-283c-457d-b298-d158475cdebe None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:06.827496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60102a1d-283c-457d-b298-d158475cdebe None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:06.827779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60102a1d-283c-457d-b298-d158475cdebe None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:06.828558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60102a1d-283c-457d-b298-d158475cdebe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:06.830060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60102a1d-283c-457d-b298-d158475cdebe None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:06.929565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 439/888] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:42:06 2022] POST /identity/v3/services => generated 269 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:06.948214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92012125-1123-44d2-83ce-acb263974374 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:06.949041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92012125-1123-44d2-83ce-acb263974374 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:06.949548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92012125-1123-44d2-83ce-acb263974374 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:06.951365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 450/889] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:06.957550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafbb50-b24c-436c-82c4-045e99acc8d8 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:06.958279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafbb50-b24c-436c-82c4-045e99acc8d8 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:06.958776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafbb50-b24c-436c-82c4-045e99acc8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:06.979167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-9cafbb50-b24c-436c-82c4-045e99acc8d8 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:06.980904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9cafbb50-b24c-436c-82c4-045e99acc8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:07.068557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 440/890] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:07.095871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92012125-1123-44d2-83ce-acb263974374 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:07.215527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92012125-1123-44d2-83ce-acb263974374 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:07.251651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efda15-c794-4993-bb9a-4f6df79991a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:07.252521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efda15-c794-4993-bb9a-4f6df79991a5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:07.252849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efda15-c794-4993-bb9a-4f6df79991a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:07.253133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efda15-c794-4993-bb9a-4f6df79991a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:07.265023 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efda15-c794-4993-bb9a-4f6df79991a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-50efda15-c794-4993-bb9a-4f6df79991a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:07.273640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:07.275219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:07.276051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 451/891] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Jul 24 19:42:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:07.285562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafbb50-b24c-436c-82c4-045e99acc8d8 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:07.378079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:07.406783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafbb50-b24c-436c-82c4-045e99acc8d8 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:07.442295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0fa6b-9e85-4ae1-a1d2-b74c3c2f3325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:07.442998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0fa6b-9e85-4ae1-a1d2-b74c3c2f3325 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:07.443290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0fa6b-9e85-4ae1-a1d2-b74c3c2f3325 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:07.443572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0fa6b-9e85-4ae1-a1d2-b74c3c2f3325 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:07.444358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0fa6b-9e85-4ae1-a1d2-b74c3c2f3325 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:07.444702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0fa6b-9e85-4ae1-a1d2-b74c3c2f3325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:07.446789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0fa6b-9e85-4ae1-a1d2-b74c3c2f3325 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:07.457189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 441/892] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Jul 24 19:42:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:07.464953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efda15-c794-4993-bb9a-4f6df79991a5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:07.537020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:07.584682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efda15-c794-4993-bb9a-4f6df79991a5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:07.617047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3b477-8cee-4fef-9c2d-056a8028d6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:07.618414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3b477-8cee-4fef-9c2d-056a8028d6fe None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:07.618704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3b477-8cee-4fef-9c2d-056a8028d6fe None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:07.618981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3b477-8cee-4fef-9c2d-056a8028d6fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:07.630453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3b477-8cee-4fef-9c2d-056a8028d6fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:07.631899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3b477-8cee-4fef-9c2d-056a8028d6fe None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-19b3b477-8cee-4fef-9c2d-056a8028d6fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:07.650496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:07.651392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:07.652277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:07.652277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:07.652277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:07.652277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:07.652277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 452/893] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:07.658286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0fa6b-9e85-4ae1-a1d2-b74c3c2f3325 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:07.770127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0fa6b-9e85-4ae1-a1d2-b74c3c2f3325 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:07.805546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b916fd8-0e71-4c70-b75a-40f1558c2c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:07.806142 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b916fd8-0e71-4c70-b75a-40f1558c2c16 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:07.806429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b916fd8-0e71-4c70-b75a-40f1558c2c16 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:07.806709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b916fd8-0e71-4c70-b75a-40f1558c2c16 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:07.807435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b916fd8-0e71-4c70-b75a-40f1558c2c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:07.807746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b916fd8-0e71-4c70-b75a-40f1558c2c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:07.809152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b916fd8-0e71-4c70-b75a-40f1558c2c16 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:07.822201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:07.826470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 442/894] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:07.827939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3b477-8cee-4fef-9c2d-056a8028d6fe None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:07.933953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3b477-8cee-4fef-9c2d-056a8028d6fe None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:07.943390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:07.971015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e355c3-6797-43dd-93ec-9df0c0ee8754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:07.971838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e355c3-6797-43dd-93ec-9df0c0ee8754 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:07.972124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e355c3-6797-43dd-93ec-9df0c0ee8754 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:07.972398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e355c3-6797-43dd-93ec-9df0c0ee8754 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:07.980731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e355c3-6797-43dd-93ec-9df0c0ee8754 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:07.982468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e355c3-6797-43dd-93ec-9df0c0ee8754 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-22e355c3-6797-43dd-93ec-9df0c0ee8754 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:42:07.997981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:07.998951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:07.999876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 453/895] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:42:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:08.005109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b916fd8-0e71-4c70-b75a-40f1558c2c16 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:08.116003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b916fd8-0e71-4c70-b75a-40f1558c2c16 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:08.153689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9186e-a17c-4701-8a7f-930a372e8c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:08.154382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9186e-a17c-4701-8a7f-930a372e8c9c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:08.154670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9186e-a17c-4701-8a7f-930a372e8c9c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:08.154967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9186e-a17c-4701-8a7f-930a372e8c9c None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:08.155865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9186e-a17c-4701-8a7f-930a372e8c9c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:08.156209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9186e-a17c-4701-8a7f-930a372e8c9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:08.157689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9186e-a17c-4701-8a7f-930a372e8c9c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:08.180784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 443/896] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Jul 24 19:42:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:08.185790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e355c3-6797-43dd-93ec-9df0c0ee8754 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:08.294644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e355c3-6797-43dd-93ec-9df0c0ee8754 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:08.334834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4dbaa-4d38-49d7-ae13-bc41ce3f3b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:08.335492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4dbaa-4d38-49d7-ae13-bc41ce3f3b3f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:08.335875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4dbaa-4d38-49d7-ae13-bc41ce3f3b3f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:08.336167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4dbaa-4d38-49d7-ae13-bc41ce3f3b3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:08.373920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:08.409437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4dbaa-4d38-49d7-ae13-bc41ce3f3b3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:08.410907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4dbaa-4d38-49d7-ae13-bc41ce3f3b3f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:08.464095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e30d6e-5f79-455e-b895-e303f2b80b96 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:08.464759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e30d6e-5f79-455e-b895-e303f2b80b96 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:08.465235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e30d6e-5f79-455e-b895-e303f2b80b96 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:08.467042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 444/897] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:08.472479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0638523-f21b-41b7-ad2f-62362e98541f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:08.473034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0638523-f21b-41b7-ad2f-62362e98541f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:08.473516 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0638523-f21b-41b7-ad2f-62362e98541f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:08.491244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f0638523-f21b-41b7-ad2f-62362e98541f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:08.493440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f0638523-f21b-41b7-ad2f-62362e98541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:08.516891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 454/898] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:08 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:42:08.525284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4dbaa-4d38-49d7-ae13-bc41ce3f3b3f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:08.570381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:08.587155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 445/899] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:08.588499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5f565-8c81-4e9a-bd49-01165bbdc798 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:08.588827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5f565-8c81-4e9a-bd49-01165bbdc798 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:08.591562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5f565-8c81-4e9a-bd49-01165bbdc798 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:08.594546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 446/900] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:08.595873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51903c76-c792-4363-9668-9bb927615312 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:08.596170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51903c76-c792-4363-9668-9bb927615312 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:08.596455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51903c76-c792-4363-9668-9bb927615312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:08.604166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:08.615313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-51903c76-c792-4363-9668-9bb927615312 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:08.616510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-51903c76-c792-4363-9668-9bb927615312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:08.641516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4dbaa-4d38-49d7-ae13-bc41ce3f3b3f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:08.677812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe17094-ed6f-4895-b9ec-9ffbc0ab57b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:08.678611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe17094-ed6f-4895-b9ec-9ffbc0ab57b5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:08.678897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe17094-ed6f-4895-b9ec-9ffbc0ab57b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:08.679179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe17094-ed6f-4895-b9ec-9ffbc0ab57b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:08.690250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 447/901] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:08.691797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51903c76-c792-4363-9668-9bb927615312 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:08.693132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe17094-ed6f-4895-b9ec-9ffbc0ab57b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:08.696854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe17094-ed6f-4895-b9ec-9ffbc0ab57b5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-dfe17094-ed6f-4895-b9ec-9ffbc0ab57b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:08.721496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:08.722489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:08.723389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:08.723389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:08.723389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:08.723389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:08.724085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 455/902] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:08.724631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe17094-ed6f-4895-b9ec-9ffbc0ab57b5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:08.802951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51903c76-c792-4363-9668-9bb927615312 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:08.821665 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:08.836293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe17094-ed6f-4895-b9ec-9ffbc0ab57b5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:08.840680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26886a2-f43d-4294-ab0c-86ec41c5f2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:08.841372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26886a2-f43d-4294-ab0c-86ec41c5f2a8 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:08.841661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26886a2-f43d-4294-ab0c-86ec41c5f2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:08.841939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26886a2-f43d-4294-ab0c-86ec41c5f2a8 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:08.842675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26886a2-f43d-4294-ab0c-86ec41c5f2a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:08.844059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26886a2-f43d-4294-ab0c-86ec41c5f2a8 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:08.857444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:08.871488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f9daf-1ee7-4ca5-8995-3621fda60c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:08.872381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f9daf-1ee7-4ca5-8995-3621fda60c50 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:08.872685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f9daf-1ee7-4ca5-8995-3621fda60c50 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:08.872966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f9daf-1ee7-4ca5-8995-3621fda60c50 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:08.873621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f9daf-1ee7-4ca5-8995-3621fda60c50 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:08.874983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f9daf-1ee7-4ca5-8995-3621fda60c50 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-073f9daf-1ee7-4ca5-8995-3621fda60c50 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:08.887834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:08.889178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 24 19:42:08.891036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:08.892903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 456/903] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:42:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:08.894234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f9daf-1ee7-4ca5-8995-3621fda60c50 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:08.909636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 448/904] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:42:08 2022] POST /identity/v3/services => generated 241 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:08.910886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26886a2-f43d-4294-ab0c-86ec41c5f2a8 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:09.041720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f9daf-1ee7-4ca5-8995-3621fda60c50 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:09.056586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26886a2-f43d-4294-ab0c-86ec41c5f2a8 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:09.080909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fe5af-e8b0-4002-ad88-4437c6c812a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:09.081583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fe5af-e8b0-4002-ad88-4437c6c812a9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.081870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fe5af-e8b0-4002-ad88-4437c6c812a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.082146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fe5af-e8b0-4002-ad88-4437c6c812a9 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.082885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fe5af-e8b0-4002-ad88-4437c6c812a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:09.083192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fe5af-e8b0-4002-ad88-4437c6c812a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:09.084718 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fe5af-e8b0-4002-ad88-4437c6c812a9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:09.099657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 457/905] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:09.100982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7e7f8-b39b-4811-98d3-24209b0889e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:09.101860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7e7f8-b39b-4811-98d3-24209b0889e6 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.102779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7e7f8-b39b-4811-98d3-24209b0889e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.103065 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7e7f8-b39b-4811-98d3-24209b0889e6 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.103789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7e7f8-b39b-4811-98d3-24209b0889e6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:09.104104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7e7f8-b39b-4811-98d3-24209b0889e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:09.105520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7e7f8-b39b-4811-98d3-24209b0889e6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:09.107889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fe5af-e8b0-4002-ad88-4437c6c812a9 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:09.111238 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 449/906] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:42:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:09.117664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7e7f8-b39b-4811-98d3-24209b0889e6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:09.163432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:09.165792 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:09.227991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fe5af-e8b0-4002-ad88-4437c6c812a9 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:09.239384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7e7f8-b39b-4811-98d3-24209b0889e6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:09.266195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e4535-e19a-4244-90fe-764c1d37a230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:09.266993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e4535-e19a-4244-90fe-764c1d37a230 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.267275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e4535-e19a-4244-90fe-764c1d37a230 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.267546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e4535-e19a-4244-90fe-764c1d37a230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.275111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e4535-e19a-4244-90fe-764c1d37a230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:09.276550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e4535-e19a-4244-90fe-764c1d37a230 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:09.276966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092e676-5464-4e51-983d-9d70d66f3ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:09.277635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092e676-5464-4e51-983d-9d70d66f3ec8 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.277923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092e676-5464-4e51-983d-9d70d66f3ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.278200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092e676-5464-4e51-983d-9d70d66f3ec8 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.278907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092e676-5464-4e51-983d-9d70d66f3ec8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:09.279217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092e676-5464-4e51-983d-9d70d66f3ec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:09.280556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092e676-5464-4e51-983d-9d70d66f3ec8 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:09.285732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 450/907] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Jul 24 19:42:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-e65e4535-e19a-4244-90fe-764c1d37a230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:09.290968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:09.292094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:09.293393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:09.293393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:09.293393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:09.293393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:09.293393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:09.293393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:09.293393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:09.293393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:09.298076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4653e-164e-4892-abd8-fa1759de10ab None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.298076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4653e-164e-4892-abd8-fa1759de10ab None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.298076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4653e-164e-4892-abd8-fa1759de10ab None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.298076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 451/908] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:09.298076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4653e-164e-4892-abd8-fa1759de10ab None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:09.298378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 458/909] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:09.298378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e4535-e19a-4244-90fe-764c1d37a230 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:09.415578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e4535-e19a-4244-90fe-764c1d37a230 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:09.417356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:09.419394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:09.423909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4653e-164e-4892-abd8-fa1759de10ab None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:09.451968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51242333-f1e3-45bd-abe4-38865dcdfaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:09.452633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51242333-f1e3-45bd-abe4-38865dcdfaa3 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.452946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51242333-f1e3-45bd-abe4-38865dcdfaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.453226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51242333-f1e3-45bd-abe4-38865dcdfaa3 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.454009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51242333-f1e3-45bd-abe4-38865dcdfaa3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:09.454318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51242333-f1e3-45bd-abe4-38865dcdfaa3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:09.455771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51242333-f1e3-45bd-abe4-38865dcdfaa3 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:09.460840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af05dc4-1851-440a-a5c9-cfbeeb7cfb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:09.462092 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af05dc4-1851-440a-a5c9-cfbeeb7cfb39 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.462377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af05dc4-1851-440a-a5c9-cfbeeb7cfb39 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.462652 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af05dc4-1851-440a-a5c9-cfbeeb7cfb39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.463413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af05dc4-1851-440a-a5c9-cfbeeb7cfb39 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:09.464823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af05dc4-1851-440a-a5c9-cfbeeb7cfb39 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:09.469387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 459/910] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:09.470893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22fd82-eec6-43d6-8733-bd4a3e72e251 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.471185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22fd82-eec6-43d6-8733-bd4a3e72e251 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.471463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22fd82-eec6-43d6-8733-bd4a3e72e251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.490143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ed22fd82-eec6-43d6-8733-bd4a3e72e251 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:09.491427 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ed22fd82-eec6-43d6-8733-bd4a3e72e251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:09.549304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 452/911] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Jul 24 19:42:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:09.561755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af05dc4-1851-440a-a5c9-cfbeeb7cfb39 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:09.572846 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 460/912] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:09.596866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22fd82-eec6-43d6-8733-bd4a3e72e251 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:09.712259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af05dc4-1851-440a-a5c9-cfbeeb7cfb39 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:09.716915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:09.733852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22fd82-eec6-43d6-8733-bd4a3e72e251 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:09.765468 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946905-65cb-4bcf-8354-f970bbc3c991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:09.766391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946905-65cb-4bcf-8354-f970bbc3c991 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.766680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946905-65cb-4bcf-8354-f970bbc3c991 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.766963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946905-65cb-4bcf-8354-f970bbc3c991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.767819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c946905-65cb-4bcf-8354-f970bbc3c991 None admin] RBAC: Adding query filter params (role.id=39f2429cfacc43b89f5973b368b61cf0, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=8fb7202aa94640129f33a310f6d00876) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:09.768137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c946905-65cb-4bcf-8354-f970bbc3c991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:09.769603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c946905-65cb-4bcf-8354-f970bbc3c991 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:09.777770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 453/913] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:09 2022] GET /identity/v3/role_assignments?user.id=8fb7202aa94640129f33a310f6d00876&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=39f2429cfacc43b89f5973b368b61cf0 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:09.785514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d533e0-1576-4689-9429-45926362f0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:09.786403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d533e0-1576-4689-9429-45926362f0e9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.786629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d533e0-1576-4689-9429-45926362f0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.786904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d533e0-1576-4689-9429-45926362f0e9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.787568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d533e0-1576-4689-9429-45926362f0e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:09.788973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d533e0-1576-4689-9429-45926362f0e9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-d7d533e0-1576-4689-9429-45926362f0e9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:09.803044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:09.804160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:09.805197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 461/914] 10.222.0.17 () {56 vars in 1132 bytes} [Sun Jul 24 19:42:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:09.812701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946905-65cb-4bcf-8354-f970bbc3c991 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:09.837276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:09.927457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946905-65cb-4bcf-8354-f970bbc3c991 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:09.971319 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c9a14-42db-4b13-b45c-8eef6690a309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:09.971920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c9a14-42db-4b13-b45c-8eef6690a309 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:09.972204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c9a14-42db-4b13-b45c-8eef6690a309 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:09.972473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c9a14-42db-4b13-b45c-8eef6690a309 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:09.973241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c9a14-42db-4b13-b45c-8eef6690a309 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:09.973564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c9a14-42db-4b13-b45c-8eef6690a309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:09.975035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c9a14-42db-4b13-b45c-8eef6690a309 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:09.981367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 454/915] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:09.987170 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d533e0-1576-4689-9429-45926362f0e9 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:10.097150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d533e0-1576-4689-9429-45926362f0e9 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:10.138273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f53d1b-405a-49c8-b5a7-4bcd2e10840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:10.139410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f53d1b-405a-49c8-b5a7-4bcd2e10840e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:10.139798 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f53d1b-405a-49c8-b5a7-4bcd2e10840e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:10.140176 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f53d1b-405a-49c8-b5a7-4bcd2e10840e None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:10.141107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f53d1b-405a-49c8-b5a7-4bcd2e10840e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:10.141526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f53d1b-405a-49c8-b5a7-4bcd2e10840e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:10.143392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f53d1b-405a-49c8-b5a7-4bcd2e10840e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:10.150590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 462/916] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:10.157840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c9a14-42db-4b13-b45c-8eef6690a309 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:10.252433 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:10.269091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c9a14-42db-4b13-b45c-8eef6690a309 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:10.306952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c7569-b8fa-493a-9051-8c54c1d0c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:10.307682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c7569-b8fa-493a-9051-8c54c1d0c3ae None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:10.307972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c7569-b8fa-493a-9051-8c54c1d0c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:10.308251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c7569-b8fa-493a-9051-8c54c1d0c3ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:10.309091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c7569-b8fa-493a-9051-8c54c1d0c3ae None admin] RBAC: Adding query filter params (interface=public, service_id=5ffa5fa57fc340e582d2cabdd2de1632, region_id=RegionOne) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:10.309406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c7569-b8fa-493a-9051-8c54c1d0c3ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:10.310806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c7569-b8fa-493a-9051-8c54c1d0c3ae None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:10.316529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 455/917] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Jul 24 19:42:10 2022] GET /identity/v3/endpoints?service_id=5ffa5fa57fc340e582d2cabdd2de1632&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:10.322655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f53d1b-405a-49c8-b5a7-4bcd2e10840e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:10.343468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:10.438842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f53d1b-405a-49c8-b5a7-4bcd2e10840e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:10.474832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca57736-870e-480c-ac99-ce9c1aea46ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:10.475506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca57736-870e-480c-ac99-ce9c1aea46ef None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:10.475788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca57736-870e-480c-ac99-ce9c1aea46ef None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:10.476061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca57736-870e-480c-ac99-ce9c1aea46ef None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:10.476773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca57736-870e-480c-ac99-ce9c1aea46ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:10.478190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca57736-870e-480c-ac99-ce9c1aea46ef None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:10.484834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 463/918] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Jul 24 19:42:10 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 19:42:11.098999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec5547-46a3-4aa5-9831-2c0eb76a6951 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:11.099540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec5547-46a3-4aa5-9831-2c0eb76a6951 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:11.099540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec5547-46a3-4aa5-9831-2c0eb76a6951 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:11.101236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 456/919] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:11.109348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f265-dcb2-4afa-bc6b-3da5060f8ecf None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:11.109833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f265-dcb2-4afa-bc6b-3da5060f8ecf None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:11.109922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f265-dcb2-4afa-bc6b-3da5060f8ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:11.129818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-add0f265-dcb2-4afa-bc6b-3da5060f8ecf None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:11.131376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-add0f265-dcb2-4afa-bc6b-3da5060f8ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:11.203266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 464/920] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:11.228812 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec5547-46a3-4aa5-9831-2c0eb76a6951 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:11.354736 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec5547-46a3-4aa5-9831-2c0eb76a6951 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:11.390290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae287ce-2af0-45f0-b424-a2c12e237026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:11.391103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae287ce-2af0-45f0-b424-a2c12e237026 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:11.391384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae287ce-2af0-45f0-b424-a2c12e237026 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:11.391659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae287ce-2af0-45f0-b424-a2c12e237026 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:11.392314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae287ce-2af0-45f0-b424-a2c12e237026 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:11.393714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae287ce-2af0-45f0-b424-a2c12e237026 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-5ae287ce-2af0-45f0-b424-a2c12e237026 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:11.408022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:11.409138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:11.409959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 457/921] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:11.414253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f265-dcb2-4afa-bc6b-3da5060f8ecf None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:11.431406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:11.522302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f265-dcb2-4afa-bc6b-3da5060f8ecf None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:11.561244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed5f87-f8d9-4873-accf-0b71eb7d440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:11.561959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5f87-f8d9-4873-accf-0b71eb7d440b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:11.562242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5f87-f8d9-4873-accf-0b71eb7d440b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:11.562515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5f87-f8d9-4873-accf-0b71eb7d440b None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:11.563232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed5f87-f8d9-4873-accf-0b71eb7d440b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:11.563538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed5f87-f8d9-4873-accf-0b71eb7d440b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:11.564945 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed5f87-f8d9-4873-accf-0b71eb7d440b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:11.571388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 465/922] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:11.577573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae287ce-2af0-45f0-b424-a2c12e237026 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:11.595228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:11.693193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae287ce-2af0-45f0-b424-a2c12e237026 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:11.731271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafcb2f-b9f9-4943-8ccb-8e74c8072df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:11.732353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafcb2f-b9f9-4943-8ccb-8e74c8072df9 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:11.732889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafcb2f-b9f9-4943-8ccb-8e74c8072df9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:11.733380 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafcb2f-b9f9-4943-8ccb-8e74c8072df9 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:11.734357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafcb2f-b9f9-4943-8ccb-8e74c8072df9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:11.734892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafcb2f-b9f9-4943-8ccb-8e74c8072df9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:11.736512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafcb2f-b9f9-4943-8ccb-8e74c8072df9 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:11.747725 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 458/923] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:11.749548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed5f87-f8d9-4873-accf-0b71eb7d440b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:11.841068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:11.853131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7da8ed-2433-4fa5-9d56-7f6ca3a9df50 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:11.853850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7da8ed-2433-4fa5-9d56-7f6ca3a9df50 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:11.854231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7da8ed-2433-4fa5-9d56-7f6ca3a9df50 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:11.856497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 459/924] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:42:11.862213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd0983-f481-4e6d-8971-08bb62dd734b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:11.862502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd0983-f481-4e6d-8971-08bb62dd734b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:11.862781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd0983-f481-4e6d-8971-08bb62dd734b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:11.877837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed5f87-f8d9-4873-accf-0b71eb7d440b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:11.883670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-47cd0983-f481-4e6d-8971-08bb62dd734b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:11.884929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-47cd0983-f481-4e6d-8971-08bb62dd734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:11.919298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b986445-abc7-40f8-973b-bcd1a97866db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:11.919966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b986445-abc7-40f8-973b-bcd1a97866db None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:11.920256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b986445-abc7-40f8-973b-bcd1a97866db None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:11.920536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b986445-abc7-40f8-973b-bcd1a97866db None admin] PATH_INFO: `/v3/endpoints` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:11.921451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b986445-abc7-40f8-973b-bcd1a97866db None admin] RBAC: Adding query filter params (interface=public, service_id=b810ce07c75542229610e371fa1712d9, region_id=RegionOne) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:11.921800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b986445-abc7-40f8-973b-bcd1a97866db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:11.923261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b986445-abc7-40f8-973b-bcd1a97866db None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:11.930835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 466/925] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Jul 24 19:42:11 2022] GET /identity/v3/endpoints?service_id=b810ce07c75542229610e371fa1712d9&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:11.938083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b986445-abc7-40f8-973b-bcd1a97866db None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:11.980690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 460/926] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:12.003280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd0983-f481-4e6d-8971-08bb62dd734b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:12.012240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:12.061564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b986445-abc7-40f8-973b-bcd1a97866db None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:12.099224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecced4-7428-48cf-a783-be5903b6f34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:12.099898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecced4-7428-48cf-a783-be5903b6f34a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.100232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecced4-7428-48cf-a783-be5903b6f34a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.100518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecced4-7428-48cf-a783-be5903b6f34a None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.101252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecced4-7428-48cf-a783-be5903b6f34a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:12.103399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecced4-7428-48cf-a783-be5903b6f34a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:12.110611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 467/927] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Jul 24 19:42:11 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 19:42:12.111593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd0983-f481-4e6d-8971-08bb62dd734b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:12.113272 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1fb70-ccb2-45ee-8b10-b71cdda830d6 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.113473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1fb70-ccb2-45ee-8b10-b71cdda830d6 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.119001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1fb70-ccb2-45ee-8b10-b71cdda830d6 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.119001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 468/928] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:12.121095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f0a12-50a9-4768-9f6b-9dc5bf128dd9 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.121617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f0a12-50a9-4768-9f6b-9dc5bf128dd9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.122130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f0a12-50a9-4768-9f6b-9dc5bf128dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.128755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:12.141969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f07f0a12-50a9-4768-9f6b-9dc5bf128dd9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:12.144401 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f07f0a12-50a9-4768-9f6b-9dc5bf128dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:12.162069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:12.163042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.163489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.163922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.165717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:12.167642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:12.170301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WARNING keystone.server.flask.common [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 24 19:42:12.219777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.sql.core [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 24 19:42:12.219777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 24 19:42:12.219777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [parameters: {'id': '5efa34001c504cf18331702ba475c93f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 24 19:42:12.219777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86806) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5efa34001c504cf18331702ba475c93f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5efa34001c504cf18331702ba475c93f. Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application result = self._query(query) Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application conn.query(q) Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 24 19:42:12.233032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application result.read() Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 24 19:42:12.233908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application next(self.gen) Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application yield resource Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application next(self.gen) Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.session.rollback() Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.force_reraise() Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise self.value Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application session.commit() Jul 24 19:42:12.234835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._prepare_impl() Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.session.flush() Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._flush(objects) Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application compat.raise_( Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application flush_context.execute() Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 24 19:42:12.235717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rec.execute(self) Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 24 19:42:12.236608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application result = self._query(query) Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application conn.query(q) Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application result.read() Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 24 19:42:12.237453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [parameters: {'id': '5efa34001c504cf18331702ba475c93f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:12.239852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:12.241739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5efa34001c504cf18331702ba475c93f. Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 461/929] 10.222.0.17 () {60 vars in 1140 bytes} [Sun Jul 24 19:42:12 2022] POST /identity/v3/projects => generated 262 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:12.243899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:12.249734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 469/930] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:12.285155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f0a12-50a9-4768-9f6b-9dc5bf128dd9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:12.331460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:12.384175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f73470-4be6-4570-8cdf-05448228f31c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:12.432675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662439b-01bd-4685-8e0c-def832210629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:12.432921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f0a12-50a9-4768-9f6b-9dc5bf128dd9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:12.433028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662439b-01bd-4685-8e0c-def832210629 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.433028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662439b-01bd-4685-8e0c-def832210629 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.433028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662439b-01bd-4685-8e0c-def832210629 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.440852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662439b-01bd-4685-8e0c-def832210629 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:12.443293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662439b-01bd-4685-8e0c-def832210629 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-1662439b-01bd-4685-8e0c-def832210629 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:12.468246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:12.469425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:12.470306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:12.470306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:12.470306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:12.470306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:12.470306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:12.470306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:12.470306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:12.470306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:12.470306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 462/931] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:12.479723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662439b-01bd-4685-8e0c-def832210629 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:12.484000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962df0a7-5317-45cd-9100-393c45aae380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:12.485406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962df0a7-5317-45cd-9100-393c45aae380 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.485908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962df0a7-5317-45cd-9100-393c45aae380 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.486395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962df0a7-5317-45cd-9100-393c45aae380 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.490023 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:12.509540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962df0a7-5317-45cd-9100-393c45aae380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-962df0a7-5317-45cd-9100-393c45aae380 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:12.527192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:12.528944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:12.530296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:12.541191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 470/932] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Jul 24 19:42:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:12.544488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962df0a7-5317-45cd-9100-393c45aae380 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:12.611254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662439b-01bd-4685-8e0c-def832210629 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:12.645117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:12.657974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6e688-4f74-4a05-a0b3-ff809b595e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:12.658755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6e688-4f74-4a05-a0b3-ff809b595e38 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.659105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6e688-4f74-4a05-a0b3-ff809b595e38 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.661402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6e688-4f74-4a05-a0b3-ff809b595e38 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.661402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6e688-4f74-4a05-a0b3-ff809b595e38 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:12.661402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6e688-4f74-4a05-a0b3-ff809b595e38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:12.662748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6e688-4f74-4a05-a0b3-ff809b595e38 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:12.683571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962df0a7-5317-45cd-9100-393c45aae380 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:12.691645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 463/933] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:42:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:12.693117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9bf8d-919a-4841-b36a-84824ff8e5e0 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.693421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9bf8d-919a-4841-b36a-84824ff8e5e0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.693707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9bf8d-919a-4841-b36a-84824ff8e5e0 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.695377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 464/934] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:12.702646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082374a-48fc-431c-b908-593a3a4507ae None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.703083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082374a-48fc-431c-b908-593a3a4507ae None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.703545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082374a-48fc-431c-b908-593a3a4507ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.722360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-0082374a-48fc-431c-b908-593a3a4507ae None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:12.724236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0082374a-48fc-431c-b908-593a3a4507ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:12.734167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d636bf-b004-44b6-8f55-bded3cd5b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:12.735916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d636bf-b004-44b6-8f55-bded3cd5b597 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.736381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d636bf-b004-44b6-8f55-bded3cd5b597 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.736855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d636bf-b004-44b6-8f55-bded3cd5b597 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.738436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d636bf-b004-44b6-8f55-bded3cd5b597 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:12.738917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d636bf-b004-44b6-8f55-bded3cd5b597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:12.741323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d636bf-b004-44b6-8f55-bded3cd5b597 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:12.753819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 471/935] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Jul 24 19:42:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:12.760320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d636bf-b004-44b6-8f55-bded3cd5b597 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:12.825280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 465/936] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:12.846581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082374a-48fc-431c-b908-593a3a4507ae None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:12.857222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:12.914870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d636bf-b004-44b6-8f55-bded3cd5b597 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:12.958190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cbe68-dd02-4599-af1b-b6fc7d65630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:12.959104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cbe68-dd02-4599-af1b-b6fc7d65630f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:12.959399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cbe68-dd02-4599-af1b-b6fc7d65630f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:12.959680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cbe68-dd02-4599-af1b-b6fc7d65630f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:12.971402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cbe68-dd02-4599-af1b-b6fc7d65630f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:12.973465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cbe68-dd02-4599-af1b-b6fc7d65630f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:12.979983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082374a-48fc-431c-b908-593a3a4507ae None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:12.994179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-ae8cbe68-dd02-4599-af1b-b6fc7d65630f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:12.994433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:12.995314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:12.996203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:12.996203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:12.996203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:12.996203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:12.996903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 472/937] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:13.002876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cbe68-dd02-4599-af1b-b6fc7d65630f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:13.024901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3be80-3654-4809-948f-fad319919aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:13.026359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3be80-3654-4809-948f-fad319919aa7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:13.026644 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3be80-3654-4809-948f-fad319919aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:13.027062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3be80-3654-4809-948f-fad319919aa7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:13.027853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3be80-3654-4809-948f-fad319919aa7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:13.029955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3be80-3654-4809-948f-fad319919aa7 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-b3f3be80-3654-4809-948f-fad319919aa7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:13.047691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:13.048719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:13.049958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 466/938] 10.222.0.17 () {56 vars in 1132 bytes} [Sun Jul 24 19:42:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:13.058564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3be80-3654-4809-948f-fad319919aa7 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:13.118421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cbe68-dd02-4599-af1b-b6fc7d65630f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:13.128678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:13.155342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f9411-2442-4557-b31d-639dfe8e576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:13.156351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f9411-2442-4557-b31d-639dfe8e576c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:13.156852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f9411-2442-4557-b31d-639dfe8e576c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:13.157306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f9411-2442-4557-b31d-639dfe8e576c None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:13.158214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f9411-2442-4557-b31d-639dfe8e576c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:13.158877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f9411-2442-4557-b31d-639dfe8e576c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:13.161431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f9411-2442-4557-b31d-639dfe8e576c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:13.176270 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 473/939] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:13.180698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3be80-3654-4809-948f-fad319919aa7 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:13.185822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f9411-2442-4557-b31d-639dfe8e576c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:13.217880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d4f78-ef81-4be8-b7e8-26e070239037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:13.219058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d4f78-ef81-4be8-b7e8-26e070239037 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:13.219634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d4f78-ef81-4be8-b7e8-26e070239037 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:13.220068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d4f78-ef81-4be8-b7e8-26e070239037 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:13.221561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d4f78-ef81-4be8-b7e8-26e070239037 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:13.222040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d4f78-ef81-4be8-b7e8-26e070239037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:13.224151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d4f78-ef81-4be8-b7e8-26e070239037 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:13.231478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 467/940] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:13.240687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d4f78-ef81-4be8-b7e8-26e070239037 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:13.300687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:13.320733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f9411-2442-4557-b31d-639dfe8e576c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:13.371401 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70632c36-79a9-4468-b780-76c02a65f52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:13.371745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d4f78-ef81-4be8-b7e8-26e070239037 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:13.372874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70632c36-79a9-4468-b780-76c02a65f52b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:13.373723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70632c36-79a9-4468-b780-76c02a65f52b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:13.375198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70632c36-79a9-4468-b780-76c02a65f52b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:13.385334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70632c36-79a9-4468-b780-76c02a65f52b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:13.387842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70632c36-79a9-4468-b780-76c02a65f52b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-70632c36-79a9-4468-b780-76c02a65f52b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:42:13.408892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:13.409894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:42:13.410832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:13.411888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac664ac-d114-4f60-9fb1-3ac318f5889a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:13.412464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac664ac-d114-4f60-9fb1-3ac318f5889a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:13.412795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac664ac-d114-4f60-9fb1-3ac318f5889a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:13.413078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac664ac-d114-4f60-9fb1-3ac318f5889a None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:13.413791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac664ac-d114-4f60-9fb1-3ac318f5889a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:13.414103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac664ac-d114-4f60-9fb1-3ac318f5889a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:13.419570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac664ac-d114-4f60-9fb1-3ac318f5889a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:13.420322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 474/941] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:42:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:13.421922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70632c36-79a9-4468-b780-76c02a65f52b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:13.428793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 468/942] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:13.430559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac664ac-d114-4f60-9fb1-3ac318f5889a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:13.458567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:13.543639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac664ac-d114-4f60-9fb1-3ac318f5889a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:13.546927 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:13.565518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70632c36-79a9-4468-b780-76c02a65f52b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:13.580462 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afbb6d-a650-4af7-92f9-1b206bd093a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:13.581392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afbb6d-a650-4af7-92f9-1b206bd093a6 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:13.581782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afbb6d-a650-4af7-92f9-1b206bd093a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:13.582228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afbb6d-a650-4af7-92f9-1b206bd093a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:13.583076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afbb6d-a650-4af7-92f9-1b206bd093a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:13.584607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afbb6d-a650-4af7-92f9-1b206bd093a6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:13.602689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1cdba-4730-46fb-a53f-2ab52363702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:13.603375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1cdba-4730-46fb-a53f-2ab52363702b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:13.603659 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1cdba-4730-46fb-a53f-2ab52363702b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:13.603938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1cdba-4730-46fb-a53f-2ab52363702b None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:13.604820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1cdba-4730-46fb-a53f-2ab52363702b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:13.605134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1cdba-4730-46fb-a53f-2ab52363702b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:13.606556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1cdba-4730-46fb-a53f-2ab52363702b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:13.624826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 475/943] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Jul 24 19:42:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:13.629981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1cdba-4730-46fb-a53f-2ab52363702b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:13.693228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 469/944] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Jul 24 19:42:13 2022] POST /identity/v3/endpoints => generated 347 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:42:13.764824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:13.794210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1cdba-4730-46fb-a53f-2ab52363702b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:13.871430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75acf-2139-451c-9762-5a46c685dbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:13.873299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75acf-2139-451c-9762-5a46c685dbd4 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:13.874112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75acf-2139-451c-9762-5a46c685dbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:13.874994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75acf-2139-451c-9762-5a46c685dbd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:13.897068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b75acf-2139-451c-9762-5a46c685dbd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:13.900954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b75acf-2139-451c-9762-5a46c685dbd4 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a2b75acf-2139-451c-9762-5a46c685dbd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:13.939278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:13.941055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:13.942792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:13.942792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:13.942792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:13.942792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:13.947533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 476/945] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:13.958968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afbb6d-a650-4af7-92f9-1b206bd093a6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:14.116572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afbb6d-a650-4af7-92f9-1b206bd093a6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:14.154675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611510e-dd8f-4c6b-b994-5905aa31dc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:14.155236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611510e-dd8f-4c6b-b994-5905aa31dc6b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:14.155530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611510e-dd8f-4c6b-b994-5905aa31dc6b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:14.155795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611510e-dd8f-4c6b-b994-5905aa31dc6b None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:14.156536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611510e-dd8f-4c6b-b994-5905aa31dc6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:14.156875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611510e-dd8f-4c6b-b994-5905aa31dc6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:14.158261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611510e-dd8f-4c6b-b994-5905aa31dc6b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:14.172176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 470/946] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:14.181201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75acf-2139-451c-9762-5a46c685dbd4 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:14.372137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75acf-2139-451c-9762-5a46c685dbd4 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:14.441125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db295b-a640-4c43-8ed8-57e51c404d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:14.442839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db295b-a640-4c43-8ed8-57e51c404d7a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:14.443612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db295b-a640-4c43-8ed8-57e51c404d7a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:14.444339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db295b-a640-4c43-8ed8-57e51c404d7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:14.448101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:14.458836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db295b-a640-4c43-8ed8-57e51c404d7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:14.464796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db295b-a640-4c43-8ed8-57e51c404d7a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-38db295b-a640-4c43-8ed8-57e51c404d7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:14.489201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:14.490156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:14.491088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:14.491088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:14.491088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:14.491088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:14.491088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:14.491088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:14.491088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:14.491088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:14.492120 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 477/947] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:14.493177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f6b4a-ebb5-49d2-a0f3-63a45cc4b0bc None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:14.493623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f6b4a-ebb5-49d2-a0f3-63a45cc4b0bc None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:14.493824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f6b4a-ebb5-49d2-a0f3-63a45cc4b0bc None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:14.496693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 471/948] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:14.501188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db295b-a640-4c43-8ed8-57e51c404d7a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:14.501378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b327ada-e303-49c7-9975-e66e1f93f096 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:14.501460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b327ada-e303-49c7-9975-e66e1f93f096 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:14.501749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b327ada-e303-49c7-9975-e66e1f93f096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:14.528509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6b327ada-e303-49c7-9975-e66e1f93f096 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:14.529746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6b327ada-e303-49c7-9975-e66e1f93f096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:14.548149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:14.617046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 472/949] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:14.636399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b327ada-e303-49c7-9975-e66e1f93f096 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:14.697345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db295b-a640-4c43-8ed8-57e51c404d7a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:14.753247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b327ada-e303-49c7-9975-e66e1f93f096 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:14.762997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48755a50-650c-4909-98d0-00fc521ea203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:14.763535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48755a50-650c-4909-98d0-00fc521ea203 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:14.764005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48755a50-650c-4909-98d0-00fc521ea203 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:14.764422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48755a50-650c-4909-98d0-00fc521ea203 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:14.765781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48755a50-650c-4909-98d0-00fc521ea203 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:14.766260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48755a50-650c-4909-98d0-00fc521ea203 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:14.768816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48755a50-650c-4909-98d0-00fc521ea203 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:14.794557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 478/950] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:14.802122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48755a50-650c-4909-98d0-00fc521ea203 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:14.803016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2e909-5db3-4c17-b4f9-7cfb0688a868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:14.804164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2e909-5db3-4c17-b4f9-7cfb0688a868 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:14.804784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2e909-5db3-4c17-b4f9-7cfb0688a868 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:14.805324 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2e909-5db3-4c17-b4f9-7cfb0688a868 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:14.806252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2e909-5db3-4c17-b4f9-7cfb0688a868 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:14.807727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2e909-5db3-4c17-b4f9-7cfb0688a868 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-2df2e909-5db3-4c17-b4f9-7cfb0688a868 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:14.821454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:14.822705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:14.823653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:14.828030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 473/951] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:14.829102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2e909-5db3-4c17-b4f9-7cfb0688a868 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:14.847741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:14.885342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:14.965635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2e909-5db3-4c17-b4f9-7cfb0688a868 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:14.985641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48755a50-650c-4909-98d0-00fc521ea203 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:15.005272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bcf3c-0ac2-4bd6-98ec-75b6dee2b756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:15.006123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bcf3c-0ac2-4bd6-98ec-75b6dee2b756 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:15.006505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bcf3c-0ac2-4bd6-98ec-75b6dee2b756 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:15.006870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bcf3c-0ac2-4bd6-98ec-75b6dee2b756 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:15.007673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bcf3c-0ac2-4bd6-98ec-75b6dee2b756 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:15.008087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bcf3c-0ac2-4bd6-98ec-75b6dee2b756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:15.009828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bcf3c-0ac2-4bd6-98ec-75b6dee2b756 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:15.020607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 474/952] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:15.021863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bcf3c-0ac2-4bd6-98ec-75b6dee2b756 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:15.044814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d619e9-ec0b-415b-aaa2-1e07cf31e8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:15.046233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d619e9-ec0b-415b-aaa2-1e07cf31e8b9 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:15.046656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d619e9-ec0b-415b-aaa2-1e07cf31e8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:15.047080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d619e9-ec0b-415b-aaa2-1e07cf31e8b9 None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/8fb7202aa94640129f33a310f6d00876/roles/39f2429cfacc43b89f5973b368b61cf0` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:15.050796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d619e9-ec0b-415b-aaa2-1e07cf31e8b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=8fb7202aa94640129f33a310f6d00876, role_id=39f2429cfacc43b89f5973b368b61cf0)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:15.053628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d619e9-ec0b-415b-aaa2-1e07cf31e8b9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:15.125354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:15.136761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 479/953] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:42:14 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/8fb7202aa94640129f33a310f6d00876/roles/39f2429cfacc43b89f5973b368b61cf0 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:42:15.140786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae661dff-ddc8-492b-8a55-7e46e6210400 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:15.141576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae661dff-ddc8-492b-8a55-7e46e6210400 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:15.142607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae661dff-ddc8-492b-8a55-7e46e6210400 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:15.146915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 480/954] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:15.157625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a9c5d-cc9f-44ea-9c52-0229de5de962 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:15.158774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a9c5d-cc9f-44ea-9c52-0229de5de962 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:15.159501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a9c5d-cc9f-44ea-9c52-0229de5de962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:15.159876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bcf3c-0ac2-4bd6-98ec-75b6dee2b756 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:15.191232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c16a9c5d-cc9f-44ea-9c52-0229de5de962 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:15.194974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c16a9c5d-cc9f-44ea-9c52-0229de5de962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:15.206030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb04ab-0868-4183-8df4-a3b4491b7a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:15.206847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb04ab-0868-4183-8df4-a3b4491b7a6f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:15.207145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb04ab-0868-4183-8df4-a3b4491b7a6f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:15.207425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb04ab-0868-4183-8df4-a3b4491b7a6f None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:15.208275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb04ab-0868-4183-8df4-a3b4491b7a6f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:15.208670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb04ab-0868-4183-8df4-a3b4491b7a6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:15.210228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb04ab-0868-4183-8df4-a3b4491b7a6f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:15.218398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 475/955] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:15.224534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb04ab-0868-4183-8df4-a3b4491b7a6f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:15.343247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 481/956] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:15.370546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a9c5d-cc9f-44ea-9c52-0229de5de962 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:15.382068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb04ab-0868-4183-8df4-a3b4491b7a6f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:15.432587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c9f6d-f1dc-4257-914e-f639bbdcf3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:15.433569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c9f6d-f1dc-4257-914e-f639bbdcf3c2 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:15.434076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c9f6d-f1dc-4257-914e-f639bbdcf3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:15.434621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c9f6d-f1dc-4257-914e-f639bbdcf3c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:15.435627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c9f6d-f1dc-4257-914e-f639bbdcf3c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:15.437352 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c9f6d-f1dc-4257-914e-f639bbdcf3c2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:15.438763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:15.504906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a9c5d-cc9f-44ea-9c52-0229de5de962 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:15.541158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 476/957] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Jul 24 19:42:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:42:15.541944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:15.562535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:15.564263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:15.564554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:15.564876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:15.565683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:15.567294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:15.568775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING keystone.server.flask.common [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 24 19:42:15.614140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.sql.core [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 24 19:42:15.614140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 24 19:42:15.614140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [parameters: {'user_id': 'bf8d9f45887542acb37f501a7606b688', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 24 19:42:15.614140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86807) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application result = self._query(query) Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application conn.query(q) Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application result.read() Jul 24 19:42:15.646364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 24 19:42:15.647333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application next(self.gen) Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application yield resource Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application next(self.gen) Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self.session.rollback() Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self.force_reraise() Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise self.value Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application session.commit() Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 24 19:42:15.648306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._prepare_impl() Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self.session.flush() Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._flush(objects) Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application compat.raise_( Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application flush_context.execute() Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rec.execute(self) Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 24 19:42:15.649305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 24 19:42:15.650202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application result = self._query(query) Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application conn.query(q) Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application result.read() Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 24 19:42:15.651055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [parameters: {'user_id': 'bf8d9f45887542acb37f501a7606b688', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:15.651982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 24 19:42:15.654105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 24 19:42:15.656269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 24 19:42:15.656269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 24 19:42:15.656269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 24 19:42:15.656269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 24 19:42:15.656269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 24 19:42:15.656269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:15.656269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 482/958] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Jul 24 19:42:15 2022] POST /identity/v3/users => generated 160 bytes in 279 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:15.660262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c9f6d-f1dc-4257-914e-f639bbdcf3c2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:15.860554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c9f6d-f1dc-4257-914e-f639bbdcf3c2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:15.921708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b655a-56d7-4bb3-a8c8-49c3a4a6248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:15.923461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b655a-56d7-4bb3-a8c8-49c3a4a6248b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:15.924356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b655a-56d7-4bb3-a8c8-49c3a4a6248b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:15.925083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b655a-56d7-4bb3-a8c8-49c3a4a6248b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:15.940158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b655a-56d7-4bb3-a8c8-49c3a4a6248b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:15.943085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b655a-56d7-4bb3-a8c8-49c3a4a6248b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-db7b655a-56d7-4bb3-a8c8-49c3a4a6248b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:42:15.971730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:15.973635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 24 19:42:15.975638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:15.984850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 477/959] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:42:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:15.991126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:16.193499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43f8b2-2716-4a07-8140-d9923ac039e1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:16.208515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:16.214967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e37f16-10af-4b7c-9581-eac7abc682b0 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:16.215286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e37f16-10af-4b7c-9581-eac7abc682b0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:16.215534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e37f16-10af-4b7c-9581-eac7abc682b0 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:16.217457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 478/960] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:42:16.223260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdb456-f204-4f90-a82c-850c162aeabe None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:16.229036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdb456-f204-4f90-a82c-850c162aeabe None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:16.229036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdb456-f204-4f90-a82c-850c162aeabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:16.242297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c7cdb456-f204-4f90-a82c-850c162aeabe None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:16.244146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c7cdb456-f204-4f90-a82c-850c162aeabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:16.257636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132fbca-8018-4ce3-a781-6b765223c122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:16.258670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132fbca-8018-4ce3-a781-6b765223c122 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:16.259083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132fbca-8018-4ce3-a781-6b765223c122 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:16.259493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132fbca-8018-4ce3-a781-6b765223c122 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:16.260871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132fbca-8018-4ce3-a781-6b765223c122 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:16.261386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132fbca-8018-4ce3-a781-6b765223c122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:16.264431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132fbca-8018-4ce3-a781-6b765223c122 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:16.318711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:16.328608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 483/961] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Jul 24 19:42:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:16.349214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 479/962] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:16.369786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132fbca-8018-4ce3-a781-6b765223c122 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:16.509527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132fbca-8018-4ce3-a781-6b765223c122 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:16.552141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48135f9-900a-41c1-9c8d-142a7a42fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:16.553054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48135f9-900a-41c1-9c8d-142a7a42fb7f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:16.553336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48135f9-900a-41c1-9c8d-142a7a42fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:16.553612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48135f9-900a-41c1-9c8d-142a7a42fb7f None admin] PATH_INFO: `/v3/services/compute` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:16.554281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48135f9-900a-41c1-9c8d-142a7a42fb7f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:16.555732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48135f9-900a-41c1-9c8d-142a7a42fb7f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a48135f9-900a-41c1-9c8d-142a7a42fb7f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:16.568801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:16.571202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:16.572990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 484/963] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:16.576703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdb456-f204-4f90-a82c-850c162aeabe None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:16.710301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdb456-f204-4f90-a82c-850c162aeabe None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:16.772174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1469cc-e628-496f-9a84-1699832e480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:16.773174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1469cc-e628-496f-9a84-1699832e480c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:16.774198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1469cc-e628-496f-9a84-1699832e480c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:16.774643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1469cc-e628-496f-9a84-1699832e480c None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:16.775846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1469cc-e628-496f-9a84-1699832e480c None admin] RBAC: Adding query filter params (name=compute) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:16.776341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1469cc-e628-496f-9a84-1699832e480c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:16.778839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1469cc-e628-496f-9a84-1699832e480c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:16.788067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 480/964] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:42:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:16.794753 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48135f9-900a-41c1-9c8d-142a7a42fb7f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:16.826482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:16.918919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48135f9-900a-41c1-9c8d-142a7a42fb7f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:16.960407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d433afe-a9f7-478a-b098-8c56982e5aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:16.961675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433afe-a9f7-478a-b098-8c56982e5aa0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:16.961962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433afe-a9f7-478a-b098-8c56982e5aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:16.962229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433afe-a9f7-478a-b098-8c56982e5aa0 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:16.962952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d433afe-a9f7-478a-b098-8c56982e5aa0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:16.963259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d433afe-a9f7-478a-b098-8c56982e5aa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:16.964694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d433afe-a9f7-478a-b098-8c56982e5aa0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:16.970692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 485/965] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:42:16 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:17.483455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea61c7d-884b-4056-bd95-2b9a3188bba2 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:17.483800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea61c7d-884b-4056-bd95-2b9a3188bba2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:17.484018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea61c7d-884b-4056-bd95-2b9a3188bba2 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:17.485503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 481/966] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:17.491489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882f570-f470-4e31-b41b-1f2dc9b4a26c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:17.492000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882f570-f470-4e31-b41b-1f2dc9b4a26c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:17.492075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882f570-f470-4e31-b41b-1f2dc9b4a26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:17.515510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d882f570-f470-4e31-b41b-1f2dc9b4a26c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:17.517270 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d882f570-f470-4e31-b41b-1f2dc9b4a26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:17.604213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 486/967] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:17.627747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea61c7d-884b-4056-bd95-2b9a3188bba2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:17.697096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:17.755880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea61c7d-884b-4056-bd95-2b9a3188bba2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:17.798415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f699a5-b0ae-4436-af1f-3b86e2d91c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:17.799622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f699a5-b0ae-4436-af1f-3b86e2d91c43 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:17.800131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f699a5-b0ae-4436-af1f-3b86e2d91c43 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:17.800672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f699a5-b0ae-4436-af1f-3b86e2d91c43 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:17.814172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f699a5-b0ae-4436-af1f-3b86e2d91c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-95f699a5-b0ae-4436-af1f-3b86e2d91c43 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:17.823662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:17.824630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:17.826302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:17.828061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 482/968] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Jul 24 19:42:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:17.839523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882f570-f470-4e31-b41b-1f2dc9b4a26c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:17.874197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:17.957102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc65288-ced1-45ea-8445-d7173629e847 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:17.957449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc65288-ced1-45ea-8445-d7173629e847 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:17.957882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc65288-ced1-45ea-8445-d7173629e847 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:17.960708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 483/969] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:17.966763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1b8c-43f1-446a-97ff-7658b4a7a4c9 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:17.967205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1b8c-43f1-446a-97ff-7658b4a7a4c9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:17.967582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1b8c-43f1-446a-97ff-7658b4a7a4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:17.997062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-57aa1b8c-43f1-446a-97ff-7658b4a7a4c9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:17.998945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-57aa1b8c-43f1-446a-97ff-7658b4a7a4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:18.023011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882f570-f470-4e31-b41b-1f2dc9b4a26c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:18.078484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f746ab6-3200-4852-9be4-dcb95636d587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:18.079536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746ab6-3200-4852-9be4-dcb95636d587 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:18.080019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746ab6-3200-4852-9be4-dcb95636d587 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:18.080488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746ab6-3200-4852-9be4-dcb95636d587 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:18.081722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f746ab6-3200-4852-9be4-dcb95636d587 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:18.082238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f746ab6-3200-4852-9be4-dcb95636d587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:18.084666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f746ab6-3200-4852-9be4-dcb95636d587 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:18.098267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 487/970] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Jul 24 19:42:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:18.104437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f746ab6-3200-4852-9be4-dcb95636d587 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:18.114041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 484/971] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:18.150325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1b8c-43f1-446a-97ff-7658b4a7a4c9 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:18.201278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:18.224283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:18.247126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f746ab6-3200-4852-9be4-dcb95636d587 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:18.278773 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1b8c-43f1-446a-97ff-7658b4a7a4c9 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:18.297554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa2400-d31f-4e69-9e36-e4209129772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:18.298376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa2400-d31f-4e69-9e36-e4209129772a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:18.298664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa2400-d31f-4e69-9e36-e4209129772a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:18.298941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa2400-d31f-4e69-9e36-e4209129772a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:18.311413 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa2400-d31f-4e69-9e36-e4209129772a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:18.313144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa2400-d31f-4e69-9e36-e4209129772a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-05aa2400-d31f-4e69-9e36-e4209129772a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:18.333061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:18.334062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:18.334950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:18.334950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:18.334950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:18.334950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:18.334950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 488/972] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:18.338824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642bc79-a25f-440a-9237-f64fa8bf320c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:18.340212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642bc79-a25f-440a-9237-f64fa8bf320c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:18.340444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa2400-d31f-4e69-9e36-e4209129772a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:18.340707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642bc79-a25f-440a-9237-f64fa8bf320c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:18.341207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642bc79-a25f-440a-9237-f64fa8bf320c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:18.342332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642bc79-a25f-440a-9237-f64fa8bf320c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:18.344836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642bc79-a25f-440a-9237-f64fa8bf320c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-8642bc79-a25f-440a-9237-f64fa8bf320c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:18.369209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:18.370860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:18.372160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 485/973] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:18.380457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642bc79-a25f-440a-9237-f64fa8bf320c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:18.476207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa2400-d31f-4e69-9e36-e4209129772a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:18.520602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:18.529661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42a81c-383c-4c48-a356-647613b563a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:18.530878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42a81c-383c-4c48-a356-647613b563a1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:18.531349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42a81c-383c-4c48-a356-647613b563a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:18.531811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42a81c-383c-4c48-a356-647613b563a1 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:18.533087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42a81c-383c-4c48-a356-647613b563a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:18.533613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42a81c-383c-4c48-a356-647613b563a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:18.536003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42a81c-383c-4c48-a356-647613b563a1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:18.558084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 489/974] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:18.567150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42a81c-383c-4c48-a356-647613b563a1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:18.572999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642bc79-a25f-440a-9237-f64fa8bf320c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:18.585825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:18.639646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1705a-6a05-41ec-a0a8-b497f4052112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:18.640443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1705a-6a05-41ec-a0a8-b497f4052112 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:18.640780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1705a-6a05-41ec-a0a8-b497f4052112 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:18.641223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1705a-6a05-41ec-a0a8-b497f4052112 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:18.642425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1705a-6a05-41ec-a0a8-b497f4052112 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:18.643072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1705a-6a05-41ec-a0a8-b497f4052112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:18.645572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1705a-6a05-41ec-a0a8-b497f4052112 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:18.655128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 486/975] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:18.661749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1705a-6a05-41ec-a0a8-b497f4052112 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:18.728336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42a81c-383c-4c48-a356-647613b563a1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:18.772358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1eccb3-3044-4683-b9c4-77a893b5d4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:18.773225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eccb3-3044-4683-b9c4-77a893b5d4da None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:18.774068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eccb3-3044-4683-b9c4-77a893b5d4da None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:18.774349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eccb3-3044-4683-b9c4-77a893b5d4da None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:18.783084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1eccb3-3044-4683-b9c4-77a893b5d4da None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:18.784538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1eccb3-3044-4683-b9c4-77a893b5d4da None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-1a1eccb3-3044-4683-b9c4-77a893b5d4da None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:42:18.800497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:18.801552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:18.803080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 490/976] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:42:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:18.808579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1eccb3-3044-4683-b9c4-77a893b5d4da None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:18.813267 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1705a-6a05-41ec-a0a8-b497f4052112 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:18.853130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bde24a-7da1-4cda-b447-88e421951254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:18.853813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bde24a-7da1-4cda-b447-88e421951254 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:18.854103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bde24a-7da1-4cda-b447-88e421951254 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:18.854385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bde24a-7da1-4cda-b447-88e421951254 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:18.855105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bde24a-7da1-4cda-b447-88e421951254 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:18.855501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bde24a-7da1-4cda-b447-88e421951254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:18.857101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bde24a-7da1-4cda-b447-88e421951254 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:18.863320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 487/977] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:18.869066 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bde24a-7da1-4cda-b447-88e421951254 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:18.874836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:18.911808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:18.959782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1eccb3-3044-4683-b9c4-77a893b5d4da None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:18.995979 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bde24a-7da1-4cda-b447-88e421951254 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:19.013694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d69f94-9159-417a-b0f6-73ebb72818f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:19.014395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d69f94-9159-417a-b0f6-73ebb72818f8 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:19.014859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d69f94-9159-417a-b0f6-73ebb72818f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:19.015031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d69f94-9159-417a-b0f6-73ebb72818f8 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:19.016133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d69f94-9159-417a-b0f6-73ebb72818f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:19.016133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d69f94-9159-417a-b0f6-73ebb72818f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:19.017700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d69f94-9159-417a-b0f6-73ebb72818f8 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:19.044841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 491/978] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Jul 24 19:42:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:19.045081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb64f62-d7b1-4b5b-bee6-6819a9e845ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:19.045081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb64f62-d7b1-4b5b-bee6-6819a9e845ab None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:19.045081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb64f62-d7b1-4b5b-bee6-6819a9e845ab None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:19.045081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb64f62-d7b1-4b5b-bee6-6819a9e845ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:19.045416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb64f62-d7b1-4b5b-bee6-6819a9e845ab None admin] RBAC: Adding query filter params (interface=admin, service_id=b810ce07c75542229610e371fa1712d9, region_id=RegionOne) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:19.045616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb64f62-d7b1-4b5b-bee6-6819a9e845ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:19.047064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d69f94-9159-417a-b0f6-73ebb72818f8 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:19.047388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb64f62-d7b1-4b5b-bee6-6819a9e845ab None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:19.053582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 488/979] 10.222.0.17 () {56 vars in 1249 bytes} [Sun Jul 24 19:42:18 2022] GET /identity/v3/endpoints?service_id=b810ce07c75542229610e371fa1712d9&interface=admin®ion_id=RegionOne => generated 193 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:19.059138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb64f62-d7b1-4b5b-bee6-6819a9e845ab None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:19.181917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:19.185886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb64f62-d7b1-4b5b-bee6-6819a9e845ab None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:19.188282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d69f94-9159-417a-b0f6-73ebb72818f8 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:19.206737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:19.229064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb26b5-2b93-4b33-8838-695ef8ade916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:19.229738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb26b5-2b93-4b33-8838-695ef8ade916 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:19.230016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb26b5-2b93-4b33-8838-695ef8ade916 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:19.230283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb26b5-2b93-4b33-8838-695ef8ade916 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:19.230968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb26b5-2b93-4b33-8838-695ef8ade916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:19.232490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb26b5-2b93-4b33-8838-695ef8ade916 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:19.248781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 489/980] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Jul 24 19:42:19 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 19:42:19.256434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531161d-d12c-4fca-950d-0db24ec1f220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:19.257655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531161d-d12c-4fca-950d-0db24ec1f220 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:19.258644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531161d-d12c-4fca-950d-0db24ec1f220 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:19.259090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531161d-d12c-4fca-950d-0db24ec1f220 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:19.330735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a649-3cca-49f9-ac05-706442a22f03 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:19.331064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a649-3cca-49f9-ac05-706442a22f03 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:19.331386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a649-3cca-49f9-ac05-706442a22f03 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:19.336702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 490/981] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:42:19.342441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0e301-01fa-4a52-9ecf-0f55d53949cb None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:19.342894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0e301-01fa-4a52-9ecf-0f55d53949cb None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:19.343349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0e301-01fa-4a52-9ecf-0f55d53949cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:19.362477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531161d-d12c-4fca-950d-0db24ec1f220 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:19.365193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531161d-d12c-4fca-950d-0db24ec1f220 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:19.366171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-86a0e301-01fa-4a52-9ecf-0f55d53949cb None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:19.367971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-86a0e301-01fa-4a52-9ecf-0f55d53949cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:19.468584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 491/982] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:19.490996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0e301-01fa-4a52-9ecf-0f55d53949cb None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:19.511004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 492/983] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:19 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 24 19:42:19.517700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531161d-d12c-4fca-950d-0db24ec1f220 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:19.530316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:19.613662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:19.625450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0e301-01fa-4a52-9ecf-0f55d53949cb None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:19.663100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531161d-d12c-4fca-950d-0db24ec1f220 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:19.666005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2209c-9906-49e0-b16f-81894e529c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:19.666966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2209c-9906-49e0-b16f-81894e529c56 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:19.667252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2209c-9906-49e0-b16f-81894e529c56 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:19.667525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2209c-9906-49e0-b16f-81894e529c56 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:19.669122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2209c-9906-49e0-b16f-81894e529c56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:19.669689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2209c-9906-49e0-b16f-81894e529c56 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:19.725889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21afec7-2d1f-4b21-ab69-533d46cd7f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:19.727310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21afec7-2d1f-4b21-ab69-533d46cd7f40 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:19.727599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21afec7-2d1f-4b21-ab69-533d46cd7f40 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:19.727876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21afec7-2d1f-4b21-ab69-533d46cd7f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:19.739734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21afec7-2d1f-4b21-ab69-533d46cd7f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:19.741275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21afec7-2d1f-4b21-ab69-533d46cd7f40 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-b21afec7-2d1f-4b21-ab69-533d46cd7f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:19.761557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:19.763656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:19.765720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:19.765720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:19.765720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:19.765720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:19.765720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 493/984] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:19.768958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21afec7-2d1f-4b21-ab69-533d46cd7f40 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:19.782593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 492/985] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:42:19 2022] POST /identity/v3/services => generated 242 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:42:19.879697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:19.931620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21afec7-2d1f-4b21-ab69-533d46cd7f40 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:19.983303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625eb998-323a-4fa9-8fdd-875d982cd47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:19.983987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625eb998-323a-4fa9-8fdd-875d982cd47b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:19.984269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625eb998-323a-4fa9-8fdd-875d982cd47b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:19.984545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625eb998-323a-4fa9-8fdd-875d982cd47b None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:19.985329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625eb998-323a-4fa9-8fdd-875d982cd47b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:19.985639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625eb998-323a-4fa9-8fdd-875d982cd47b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:19.987031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625eb998-323a-4fa9-8fdd-875d982cd47b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:20.001607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 494/986] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:20.007483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2209c-9906-49e0-b16f-81894e529c56 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:20.151618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2209c-9906-49e0-b16f-81894e529c56 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:20.192912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11637b-d80b-41c3-92a8-eb4ef2ff2ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:20.193735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11637b-d80b-41c3-92a8-eb4ef2ff2ec8 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:20.194034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11637b-d80b-41c3-92a8-eb4ef2ff2ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:20.194297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11637b-d80b-41c3-92a8-eb4ef2ff2ec8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:20.203185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11637b-d80b-41c3-92a8-eb4ef2ff2ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:20.204832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11637b-d80b-41c3-92a8-eb4ef2ff2ec8 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-1b11637b-d80b-41c3-92a8-eb4ef2ff2ec8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:20.219744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:20.221028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:20.222311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:20.222311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:20.222311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:20.222311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:20.222311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:20.222311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:20.222311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:20.222311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:20.222311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 493/987] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:20.231125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625eb998-323a-4fa9-8fdd-875d982cd47b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:20.395647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625eb998-323a-4fa9-8fdd-875d982cd47b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:20.417808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:20.442514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467e3ab-6ac5-489f-94ba-bed3a44d0235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:20.443204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467e3ab-6ac5-489f-94ba-bed3a44d0235 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:20.443499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467e3ab-6ac5-489f-94ba-bed3a44d0235 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:20.443783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467e3ab-6ac5-489f-94ba-bed3a44d0235 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:20.444591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467e3ab-6ac5-489f-94ba-bed3a44d0235 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:20.444947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467e3ab-6ac5-489f-94ba-bed3a44d0235 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:20.447011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467e3ab-6ac5-489f-94ba-bed3a44d0235 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:20.465882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 495/988] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:20.467809 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11637b-d80b-41c3-92a8-eb4ef2ff2ec8 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:20.484845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:20.623136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11637b-d80b-41c3-92a8-eb4ef2ff2ec8 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:20.684697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ede10b-0aeb-48f4-8c7f-735ae9a49380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:20.685780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ede10b-0aeb-48f4-8c7f-735ae9a49380 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:20.686286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ede10b-0aeb-48f4-8c7f-735ae9a49380 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:20.686780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ede10b-0aeb-48f4-8c7f-735ae9a49380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:20.688454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ede10b-0aeb-48f4-8c7f-735ae9a49380 None admin] RBAC: Adding query filter params (role.id=39f2429cfacc43b89f5973b368b61cf0, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=8fb7202aa94640129f33a310f6d00876) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:20.689047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ede10b-0aeb-48f4-8c7f-735ae9a49380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:20.691851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ede10b-0aeb-48f4-8c7f-735ae9a49380 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:20.705757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 494/989] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:20 2022] GET /identity/v3/role_assignments?user.id=8fb7202aa94640129f33a310f6d00876&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=39f2429cfacc43b89f5973b368b61cf0 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:21.506342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb7999-3c17-4fe6-91d9-f0677fdd536d None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:21.507106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb7999-3c17-4fe6-91d9-f0677fdd536d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:21.507285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb7999-3c17-4fe6-91d9-f0677fdd536d None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:21.512713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 496/990] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:21.515018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e0e04-ca70-4b31-a1d8-82741d7ce00d None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:21.515463 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e0e04-ca70-4b31-a1d8-82741d7ce00d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:21.515882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e0e04-ca70-4b31-a1d8-82741d7ce00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:21.536470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-906e0e04-ca70-4b31-a1d8-82741d7ce00d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:21.539551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-906e0e04-ca70-4b31-a1d8-82741d7ce00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:21.668978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 495/991] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:21.703449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb7999-3c17-4fe6-91d9-f0677fdd536d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:21.813127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb7999-3c17-4fe6-91d9-f0677fdd536d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:21.850135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d793c-e578-40a1-bf29-239faec81fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:21.850951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d793c-e578-40a1-bf29-239faec81fa6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:21.851242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d793c-e578-40a1-bf29-239faec81fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:21.852095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d793c-e578-40a1-bf29-239faec81fa6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:21.852778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d793c-e578-40a1-bf29-239faec81fa6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:21.854171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d793c-e578-40a1-bf29-239faec81fa6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-269d793c-e578-40a1-bf29-239faec81fa6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:21.866594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:21.867630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:21.873950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 497/992] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Jul 24 19:42:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:21.876311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e0e04-ca70-4b31-a1d8-82741d7ce00d None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:21.884727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:21.988297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e0e04-ca70-4b31-a1d8-82741d7ce00d None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:22.026102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c04b-62a1-43d5-abe9-36ac0f14b16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:22.026788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c04b-62a1-43d5-abe9-36ac0f14b16d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.027074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c04b-62a1-43d5-abe9-36ac0f14b16d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.027353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c04b-62a1-43d5-abe9-36ac0f14b16d None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.028079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c04b-62a1-43d5-abe9-36ac0f14b16d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:22.028399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c04b-62a1-43d5-abe9-36ac0f14b16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:22.029823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c04b-62a1-43d5-abe9-36ac0f14b16d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:22.040202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 496/993] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:22.042077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d793c-e578-40a1-bf29-239faec81fa6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:22.058762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f75997-64a5-4f9c-95c3-0669d3071d55 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.059281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f75997-64a5-4f9c-95c3-0669d3071d55 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.059780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f75997-64a5-4f9c-95c3-0669d3071d55 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.062744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 497/994] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:22.069911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1962fc-4270-4e66-8da3-07363a852099 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.070968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1962fc-4270-4e66-8da3-07363a852099 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.071442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1962fc-4270-4e66-8da3-07363a852099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.094390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:22.101779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-eb1962fc-4270-4e66-8da3-07363a852099 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:22.103867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-eb1962fc-4270-4e66-8da3-07363a852099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:22.161986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d793c-e578-40a1-bf29-239faec81fa6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:22.189746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:22.196701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 498/995] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:22.202502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fe53c-a79e-45ba-815e-ad8b7e0aa965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:22.203177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fe53c-a79e-45ba-815e-ad8b7e0aa965 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.203462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fe53c-a79e-45ba-815e-ad8b7e0aa965 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.203739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fe53c-a79e-45ba-815e-ad8b7e0aa965 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.204480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fe53c-a79e-45ba-815e-ad8b7e0aa965 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:22.204836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fe53c-a79e-45ba-815e-ad8b7e0aa965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:22.206215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fe53c-a79e-45ba-815e-ad8b7e0aa965 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:22.212592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 498/996] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Jul 24 19:42:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:22.219174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1962fc-4270-4e66-8da3-07363a852099 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:22.221039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fe53c-a79e-45ba-815e-ad8b7e0aa965 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:22.340126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fe53c-a79e-45ba-815e-ad8b7e0aa965 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:22.348548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:22.381013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2d1ca-aa59-43fb-969c-cfdb5bab05d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:22.381825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2d1ca-aa59-43fb-969c-cfdb5bab05d5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.382110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2d1ca-aa59-43fb-969c-cfdb5bab05d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.382387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2d1ca-aa59-43fb-969c-cfdb5bab05d5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.383613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2d1ca-aa59-43fb-969c-cfdb5bab05d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:22.385143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2d1ca-aa59-43fb-969c-cfdb5bab05d5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:22.388427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1962fc-4270-4e66-8da3-07363a852099 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-86c2d1ca-aa59-43fb-969c-cfdb5bab05d5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:22.399603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:22.401462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:22.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 499/997] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:22.407191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2d1ca-aa59-43fb-969c-cfdb5bab05d5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:22.443649 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1adc3-4094-4f07-b86f-298a5e65c497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:22.444743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1adc3-4094-4f07-b86f-298a5e65c497 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.445758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1adc3-4094-4f07-b86f-298a5e65c497 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.446215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1adc3-4094-4f07-b86f-298a5e65c497 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.447460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1adc3-4094-4f07-b86f-298a5e65c497 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:22.449902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1adc3-4094-4f07-b86f-298a5e65c497 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:22.528769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 499/998] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Jul 24 19:42:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:22.533954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2d1ca-aa59-43fb-969c-cfdb5bab05d5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:22.605897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854d57b-ac03-4d0c-a9b2-f0da8add7dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:22.606824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854d57b-ac03-4d0c-a9b2-f0da8add7dbb None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.607256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854d57b-ac03-4d0c-a9b2-f0da8add7dbb None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.607813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854d57b-ac03-4d0c-a9b2-f0da8add7dbb None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.609087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854d57b-ac03-4d0c-a9b2-f0da8add7dbb None admin] RBAC: Adding query filter params (name=compute) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:22.609567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854d57b-ac03-4d0c-a9b2-f0da8add7dbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:22.612150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854d57b-ac03-4d0c-a9b2-f0da8add7dbb None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:22.626255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 500/999] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:42:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:22.627627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1adc3-4094-4f07-b86f-298a5e65c497 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:22.703197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:22.755392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1adc3-4094-4f07-b86f-298a5e65c497 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:22.798743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774fa2d-888d-4840-ab07-d1723c96b27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:22.799433 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774fa2d-888d-4840-ab07-d1723c96b27f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.799721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774fa2d-888d-4840-ab07-d1723c96b27f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.800007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774fa2d-888d-4840-ab07-d1723c96b27f None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.800766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774fa2d-888d-4840-ab07-d1723c96b27f None admin] RBAC: Adding query filter params (type=compute) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:22.801079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774fa2d-888d-4840-ab07-d1723c96b27f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:22.802464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774fa2d-888d-4840-ab07-d1723c96b27f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:22.808247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 500/1000] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:42:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:22.814989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854d57b-ac03-4d0c-a9b2-f0da8add7dbb None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:22.870447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:22.950260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeb02f-b15d-4a69-8a36-31759f658730 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.950938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeb02f-b15d-4a69-8a36-31759f658730 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.950938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeb02f-b15d-4a69-8a36-31759f658730 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.952405 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 501/1001] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:42:22.957870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6be23-ba8b-4b6e-916e-deee1cdf84e7 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:22.958199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6be23-ba8b-4b6e-916e-deee1cdf84e7 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:22.958443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6be23-ba8b-4b6e-916e-deee1cdf84e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:22.977738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-04a6be23-ba8b-4b6e-916e-deee1cdf84e7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:22.978971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-04a6be23-ba8b-4b6e-916e-deee1cdf84e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:23.003608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854d57b-ac03-4d0c-a9b2-f0da8add7dbb None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:23.050838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac58f7-9773-401c-8531-cb19841a3f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:23.051504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac58f7-9773-401c-8531-cb19841a3f23 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:23.051801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac58f7-9773-401c-8531-cb19841a3f23 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:23.052082 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac58f7-9773-401c-8531-cb19841a3f23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:23.052922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac58f7-9773-401c-8531-cb19841a3f23 None admin] RBAC: Adding query filter params (interface=public, service_id=39f6d75e1d9a4bcc890560bd0dd30727, region_id=RegionOne) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:23.053252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac58f7-9773-401c-8531-cb19841a3f23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:23.054692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac58f7-9773-401c-8531-cb19841a3f23 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:23.060916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 501/1002] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Jul 24 19:42:22 2022] GET /identity/v3/endpoints?service_id=39f6d75e1d9a4bcc890560bd0dd30727&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:23.064211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 502/1003] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:23.066287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac58f7-9773-401c-8531-cb19841a3f23 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:23.087758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6be23-ba8b-4b6e-916e-deee1cdf84e7 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:23.166628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:23.177225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:23.184570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac58f7-9773-401c-8531-cb19841a3f23 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:23.226520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6be23-ba8b-4b6e-916e-deee1cdf84e7 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:23.234541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc944676-8133-4359-a014-1487ee11817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:23.235607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc944676-8133-4359-a014-1487ee11817e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:23.236085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc944676-8133-4359-a014-1487ee11817e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:23.236545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc944676-8133-4359-a014-1487ee11817e None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:23.237788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc944676-8133-4359-a014-1487ee11817e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:23.240203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc944676-8133-4359-a014-1487ee11817e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:23.250225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 502/1004] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Jul 24 19:42:23 2022] GET /identity/v3/services => generated 1727 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 19:42:23.283885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de2753-ff1b-447e-8662-6878694d1a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:23.285197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de2753-ff1b-447e-8662-6878694d1a6e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:23.285728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de2753-ff1b-447e-8662-6878694d1a6e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:23.286235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de2753-ff1b-447e-8662-6878694d1a6e None admin] PATH_INFO: `/v3/services/image` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:23.287150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de2753-ff1b-447e-8662-6878694d1a6e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:23.288925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de2753-ff1b-447e-8662-6878694d1a6e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-b1de2753-ff1b-447e-8662-6878694d1a6e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:23.302869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:23.304159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 24 19:42:23.306396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:23.308869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 503/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Sun Jul 24 19:42:23 2022] GET /identity/v3/services/image => generated 86 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:23.314707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc944676-8133-4359-a014-1487ee11817e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:23.465077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc944676-8133-4359-a014-1487ee11817e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:23.505095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0042fd-04ad-4985-9430-062d26ebe63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:23.505773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0042fd-04ad-4985-9430-062d26ebe63e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:23.506063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0042fd-04ad-4985-9430-062d26ebe63e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:23.506345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0042fd-04ad-4985-9430-062d26ebe63e None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:23.507065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0042fd-04ad-4985-9430-062d26ebe63e None admin] RBAC: Adding query filter params (name=image) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:23.507379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0042fd-04ad-4985-9430-062d26ebe63e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:23.508847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0042fd-04ad-4985-9430-062d26ebe63e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:23.520679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 503/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:23.524409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de2753-ff1b-447e-8662-6878694d1a6e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:23.723314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de2753-ff1b-447e-8662-6878694d1a6e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:23.757777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:23.789639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18172c-0edc-461a-996b-68ed93c3b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:23.790801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18172c-0edc-461a-996b-68ed93c3b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:23.791562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18172c-0edc-461a-996b-68ed93c3b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:23.792067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18172c-0edc-461a-996b-68ed93c3b8c5 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:23.793444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18172c-0edc-461a-996b-68ed93c3b8c5 None admin] RBAC: Adding query filter params (type=image) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:23.794010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18172c-0edc-461a-996b-68ed93c3b8c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:23.796769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18172c-0edc-461a-996b-68ed93c3b8c5 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:23.812689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 504/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:23 2022] GET /identity/v3/services?type=image => generated 122 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:24.811917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341b73d-9044-425e-8df8-484c0ad0951d None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:24.812755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341b73d-9044-425e-8df8-484c0ad0951d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:24.812894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341b73d-9044-425e-8df8-484c0ad0951d None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:24.814476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 504/1008] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:24.820693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb82d-9275-4089-87ae-d3569fa0589f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:24.821395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb82d-9275-4089-87ae-d3569fa0589f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:24.821661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb82d-9275-4089-87ae-d3569fa0589f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:24.846093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-254eb82d-9275-4089-87ae-d3569fa0589f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:24.848085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-254eb82d-9275-4089-87ae-d3569fa0589f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:24.971841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 505/1009] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:24.993066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341b73d-9044-425e-8df8-484c0ad0951d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:25.022724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:25.156307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341b73d-9044-425e-8df8-484c0ad0951d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:25.217607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdccdfd-6406-489f-b906-7ff166021d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:25.218992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdccdfd-6406-489f-b906-7ff166021d78 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:25.219502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdccdfd-6406-489f-b906-7ff166021d78 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:25.220004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdccdfd-6406-489f-b906-7ff166021d78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:25.222142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdccdfd-6406-489f-b906-7ff166021d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:25.224874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdccdfd-6406-489f-b906-7ff166021d78 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:25.233152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 505/1010] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:25.234949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254eb82d-9275-4089-87ae-d3569fa0589f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:25.391877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254eb82d-9275-4089-87ae-d3569fa0589f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:25.417880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:25.449686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72141c5-f261-490c-95bb-65bd733b77fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:25.449995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72141c5-f261-490c-95bb-65bd733b77fb None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:25.450131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72141c5-f261-490c-95bb-65bd733b77fb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:25.450447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72141c5-f261-490c-95bb-65bd733b77fb None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:25.452781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72141c5-f261-490c-95bb-65bd733b77fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:25.452781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72141c5-f261-490c-95bb-65bd733b77fb None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:25.538429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 506/1011] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:42:25 2022] POST /identity/v3/projects => generated 311 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:25.675614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc9473-37f3-4921-97d9-baa132cd3c9f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:25.675888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc9473-37f3-4921-97d9-baa132cd3c9f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:25.676080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc9473-37f3-4921-97d9-baa132cd3c9f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:25.677859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 506/1012] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:42:25.684601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2893c-9516-402e-99dd-0a482d6145a0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:25.685554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2893c-9516-402e-99dd-0a482d6145a0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:25.685554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2893c-9516-402e-99dd-0a482d6145a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:25.705175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e9f2893c-9516-402e-99dd-0a482d6145a0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:25.706390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e9f2893c-9516-402e-99dd-0a482d6145a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:25.787536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 507/1013] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:25.812790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc9473-37f3-4921-97d9-baa132cd3c9f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:25.916353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:25.928053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc9473-37f3-4921-97d9-baa132cd3c9f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:25.967635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a0d84-8140-4533-9c63-f9ba58595ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:25.968443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0d84-8140-4533-9c63-f9ba58595ada None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:25.968763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0d84-8140-4533-9c63-f9ba58595ada None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:25.969039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0d84-8140-4533-9c63-f9ba58595ada None admin] PATH_INFO: `/v3/services/compute` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:25.969754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a0d84-8140-4533-9c63-f9ba58595ada None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:25.971593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a0d84-8140-4533-9c63-f9ba58595ada None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-fb5a0d84-8140-4533-9c63-f9ba58595ada None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:25.984768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:25.986739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 24 19:42:25.988293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:25.992028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 507/1014] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:25.992418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2893c-9516-402e-99dd-0a482d6145a0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:26.100057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:26.102140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e88a0-ab5e-4a76-aca3-09c71f19a404 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:26.102430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e88a0-ab5e-4a76-aca3-09c71f19a404 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:26.102711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e88a0-ab5e-4a76-aca3-09c71f19a404 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:26.104141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 508/1015] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:26.111145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d18c6-2cfa-46ed-978f-0c77e8c2b946 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:26.113416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2893c-9516-402e-99dd-0a482d6145a0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:26.113572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d18c6-2cfa-46ed-978f-0c77e8c2b946 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:26.113572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d18c6-2cfa-46ed-978f-0c77e8c2b946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:26.192354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-6a2d18c6-2cfa-46ed-978f-0c77e8c2b946 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:26.193860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6a2d18c6-2cfa-46ed-978f-0c77e8c2b946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:26.219640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2178c-075d-414b-a4bc-bb22acd988fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:26.220635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2178c-075d-414b-a4bc-bb22acd988fd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:26.221099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2178c-075d-414b-a4bc-bb22acd988fd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:26.221527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2178c-075d-414b-a4bc-bb22acd988fd None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:26.222398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2178c-075d-414b-a4bc-bb22acd988fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:26.222882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2178c-075d-414b-a4bc-bb22acd988fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:26.224405 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2178c-075d-414b-a4bc-bb22acd988fd None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:26.230899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 508/1016] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:42:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:26.237186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2178c-075d-414b-a4bc-bb22acd988fd None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:26.290972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 509/1017] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:26.330535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d18c6-2cfa-46ed-978f-0c77e8c2b946 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:26.395387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2178c-075d-414b-a4bc-bb22acd988fd None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:26.458978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928759b-ea7a-403f-8711-22f2820e1337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:26.460012 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928759b-ea7a-403f-8711-22f2820e1337 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:26.460528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928759b-ea7a-403f-8711-22f2820e1337 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:26.461077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928759b-ea7a-403f-8711-22f2820e1337 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:26.462429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928759b-ea7a-403f-8711-22f2820e1337 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:26.463010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928759b-ea7a-403f-8711-22f2820e1337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:26.466122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928759b-ea7a-403f-8711-22f2820e1337 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:26.474690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 509/1018] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Jul 24 19:42:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:26.479540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:26.484417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928759b-ea7a-403f-8711-22f2820e1337 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:26.505998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d18c6-2cfa-46ed-978f-0c77e8c2b946 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:26.524568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:26.563737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed87524-44de-4f84-9def-50fa984487c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:26.564466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed87524-44de-4f84-9def-50fa984487c8 None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:26.564930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed87524-44de-4f84-9def-50fa984487c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:26.565393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed87524-44de-4f84-9def-50fa984487c8 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:26.566674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed87524-44de-4f84-9def-50fa984487c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:26.569029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed87524-44de-4f84-9def-50fa984487c8 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:26.624800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928759b-ea7a-403f-8711-22f2820e1337 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:26.656712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 510/1019] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:42:26 2022] POST /identity/v3/services => generated 242 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:42:26.680351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15cf36-cfb1-41f9-a1eb-6c4b2a8d4429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:26.681563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15cf36-cfb1-41f9-a1eb-6c4b2a8d4429 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:26.682072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15cf36-cfb1-41f9-a1eb-6c4b2a8d4429 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:26.682593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15cf36-cfb1-41f9-a1eb-6c4b2a8d4429 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:26.683733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db15cf36-cfb1-41f9-a1eb-6c4b2a8d4429 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:26.685521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db15cf36-cfb1-41f9-a1eb-6c4b2a8d4429 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:26.784858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 510/1020] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Jul 24 19:42:26 2022] POST /identity/v3/endpoints => generated 334 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:28.152913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceeb9a-2721-4afb-a943-21da97d93499 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:28.153533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceeb9a-2721-4afb-a943-21da97d93499 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:28.153533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceeb9a-2721-4afb-a943-21da97d93499 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:28.156011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 511/1021] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:28.160683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421028af-21b5-4a21-bf95-3a09dd819eda None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:28.161122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421028af-21b5-4a21-bf95-3a09dd819eda None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:28.161344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421028af-21b5-4a21-bf95-3a09dd819eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:28.179618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-421028af-21b5-4a21-bf95-3a09dd819eda None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:28.180874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-421028af-21b5-4a21-bf95-3a09dd819eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:28.253197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:28.260673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 511/1022] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:28.284586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceeb9a-2721-4afb-a943-21da97d93499 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:28.391739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceeb9a-2721-4afb-a943-21da97d93499 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:28.432975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee3bcd-a69d-477e-bf61-3a59989c888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:28.433792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee3bcd-a69d-477e-bf61-3a59989c888f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:28.434078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee3bcd-a69d-477e-bf61-3a59989c888f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:28.434352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee3bcd-a69d-477e-bf61-3a59989c888f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:28.435546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee3bcd-a69d-477e-bf61-3a59989c888f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:28.437013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee3bcd-a69d-477e-bf61-3a59989c888f None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:28.443262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 512/1023] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:28.445456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421028af-21b5-4a21-bf95-3a09dd819eda None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:28.565282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421028af-21b5-4a21-bf95-3a09dd819eda None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:28.602149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4abaf2-7688-492c-add8-a061a8fbe3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:28.602853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4abaf2-7688-492c-add8-a061a8fbe3f5 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:28.603143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4abaf2-7688-492c-add8-a061a8fbe3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:28.603424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4abaf2-7688-492c-add8-a061a8fbe3f5 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:28.604165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4abaf2-7688-492c-add8-a061a8fbe3f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:28.605670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4abaf2-7688-492c-add8-a061a8fbe3f5 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:28.716766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 512/1024] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:42:28 2022] POST /identity/v3/users => generated 285 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:28.977569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714494f4-10c7-4756-8884-543fdb41a199 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:28.977876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714494f4-10c7-4756-8884-543fdb41a199 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:28.978146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714494f4-10c7-4756-8884-543fdb41a199 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:28.979543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 513/1025] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:28.985201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022f974-d3f4-481f-9bff-87daedd76f7e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:28.985505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022f974-d3f4-481f-9bff-87daedd76f7e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:28.985795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022f974-d3f4-481f-9bff-87daedd76f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:29.003437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-0022f974-d3f4-481f-9bff-87daedd76f7e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:29.004666 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0022f974-d3f4-481f-9bff-87daedd76f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:29.059727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:29.077268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 513/1026] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:29.101828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714494f4-10c7-4756-8884-543fdb41a199 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:29.118119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:29.216835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714494f4-10c7-4756-8884-543fdb41a199 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:29.230126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ec476-07b0-40f3-a7fc-322b2a37ab59 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:29.230643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ec476-07b0-40f3-a7fc-322b2a37ab59 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:29.231044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ec476-07b0-40f3-a7fc-322b2a37ab59 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:29.234192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 514/1027] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:29.239873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146aeb60-dbbb-428e-8718-03096066b711 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:29.240333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146aeb60-dbbb-428e-8718-03096066b711 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:29.240609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146aeb60-dbbb-428e-8718-03096066b711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:29.264013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-146aeb60-dbbb-428e-8718-03096066b711 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:29.266455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-146aeb60-dbbb-428e-8718-03096066b711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:29.274419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb4a41-4e35-4e3e-9fb1-d189e19fbea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:29.275858 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb4a41-4e35-4e3e-9fb1-d189e19fbea6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:29.276367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb4a41-4e35-4e3e-9fb1-d189e19fbea6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:29.276910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb4a41-4e35-4e3e-9fb1-d189e19fbea6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:29.297094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb4a41-4e35-4e3e-9fb1-d189e19fbea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-93cb4a41-4e35-4e3e-9fb1-d189e19fbea6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:29.314089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:29.315081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:29.315926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:29.316567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 514/1028] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Jul 24 19:42:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:29.322426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb4a41-4e35-4e3e-9fb1-d189e19fbea6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:29.373064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 515/1029] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:29.393786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146aeb60-dbbb-428e-8718-03096066b711 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:29.417152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:29.465255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb4a41-4e35-4e3e-9fb1-d189e19fbea6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:29.519790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c6c67-20f0-433e-9279-a876c08f80f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:29.520389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c6c67-20f0-433e-9279-a876c08f80f3 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:29.520815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c6c67-20f0-433e-9279-a876c08f80f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:29.521097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c6c67-20f0-433e-9279-a876c08f80f3 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:29.521873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c6c67-20f0-433e-9279-a876c08f80f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:29.522185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c6c67-20f0-433e-9279-a876c08f80f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:29.524152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c6c67-20f0-433e-9279-a876c08f80f3 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:29.533241 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 515/1030] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Jul 24 19:42:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:29.539066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c6c67-20f0-433e-9279-a876c08f80f3 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:29.553547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146aeb60-dbbb-428e-8718-03096066b711 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:29.579887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:29.601441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03638cd-0d6c-4fb2-bf05-6440137a4409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:29.602890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03638cd-0d6c-4fb2-bf05-6440137a4409 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:29.603414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03638cd-0d6c-4fb2-bf05-6440137a4409 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:29.603919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03638cd-0d6c-4fb2-bf05-6440137a4409 None admin] PATH_INFO: `/v3/services/image` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:29.605274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03638cd-0d6c-4fb2-bf05-6440137a4409 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:29.607968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03638cd-0d6c-4fb2-bf05-6440137a4409 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-c03638cd-0d6c-4fb2-bf05-6440137a4409 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:29.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:29.628385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:29.630120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 516/1031] 10.222.0.17 () {56 vars in 1105 bytes} [Sun Jul 24 19:42:29 2022] GET /identity/v3/services/image => generated 86 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:29.634690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03638cd-0d6c-4fb2-bf05-6440137a4409 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:29.662607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c6c67-20f0-433e-9279-a876c08f80f3 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:29.698522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfc318-96ba-4629-aaf6-a550fb120063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:29.699332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfc318-96ba-4629-aaf6-a550fb120063 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:29.699619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfc318-96ba-4629-aaf6-a550fb120063 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:29.699901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfc318-96ba-4629-aaf6-a550fb120063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:29.702162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:29.712495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfc318-96ba-4629-aaf6-a550fb120063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:29.715054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfc318-96ba-4629-aaf6-a550fb120063 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-cecfc318-96ba-4629-aaf6-a550fb120063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:29.748038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:29.749053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:29.749983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:29.749983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:29.749983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:29.749983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:29.750816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 516/1032] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:29.758992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfc318-96ba-4629-aaf6-a550fb120063 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:29.812180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03638cd-0d6c-4fb2-bf05-6440137a4409 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:29.869181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb58f-55b8-4ee0-83b9-65f0ebe2a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:29.869990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb58f-55b8-4ee0-83b9-65f0ebe2a66a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:29.870465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb58f-55b8-4ee0-83b9-65f0ebe2a66a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:29.870923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb58f-55b8-4ee0-83b9-65f0ebe2a66a None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:29.872154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abb58f-55b8-4ee0-83b9-65f0ebe2a66a None admin] RBAC: Adding query filter params (name=image) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:29.872628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abb58f-55b8-4ee0-83b9-65f0ebe2a66a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:29.875000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abb58f-55b8-4ee0-83b9-65f0ebe2a66a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:29.888496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 517/1033] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:29.889970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb58f-55b8-4ee0-83b9-65f0ebe2a66a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:29.934003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfc318-96ba-4629-aaf6-a550fb120063 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:29.970705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:29.991413 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcc995-6c5c-4676-9af1-9664ae6c0a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:29.993041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcc995-6c5c-4676-9af1-9664ae6c0a15 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:29.993516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcc995-6c5c-4676-9af1-9664ae6c0a15 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:29.994241 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcc995-6c5c-4676-9af1-9664ae6c0a15 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:29.995104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcc995-6c5c-4676-9af1-9664ae6c0a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:29.995608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcc995-6c5c-4676-9af1-9664ae6c0a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:29.997951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcc995-6c5c-4676-9af1-9664ae6c0a15 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:30.014221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 517/1034] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:30.020232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcc995-6c5c-4676-9af1-9664ae6c0a15 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:30.052368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb58f-55b8-4ee0-83b9-65f0ebe2a66a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:30.079839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:30.095117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172ffc-f040-4cc8-94fa-c19d6ae2d45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:30.095904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172ffc-f040-4cc8-94fa-c19d6ae2d45a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:30.096311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172ffc-f040-4cc8-94fa-c19d6ae2d45a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:30.096791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172ffc-f040-4cc8-94fa-c19d6ae2d45a None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:30.097671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d172ffc-f040-4cc8-94fa-c19d6ae2d45a None admin] RBAC: Adding query filter params (type=image) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:30.098078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d172ffc-f040-4cc8-94fa-c19d6ae2d45a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:30.099671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d172ffc-f040-4cc8-94fa-c19d6ae2d45a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:30.108708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 518/1035] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:30.112042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172ffc-f040-4cc8-94fa-c19d6ae2d45a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:30.150747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcc995-6c5c-4676-9af1-9664ae6c0a15 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:30.195248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e952-7f97-4a89-bdaa-cbd962b70859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:30.196365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e952-7f97-4a89-bdaa-cbd962b70859 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:30.196821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e952-7f97-4a89-bdaa-cbd962b70859 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:30.197292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e952-7f97-4a89-bdaa-cbd962b70859 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:30.206592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e952-7f97-4a89-bdaa-cbd962b70859 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:30.208375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e952-7f97-4a89-bdaa-cbd962b70859 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-abf8e952-7f97-4a89-bdaa-cbd962b70859 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:42:30.228428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:30.230071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:42:30.231179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:30.236340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 518/1036] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Jul 24 19:42:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:30.237505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e952-7f97-4a89-bdaa-cbd962b70859 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:30.239109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172ffc-f040-4cc8-94fa-c19d6ae2d45a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:30.252452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:30.282375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fde67-1c77-43d8-a6ea-a559c4ee9fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:30.283086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fde67-1c77-43d8-a6ea-a559c4ee9fbd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:30.283370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fde67-1c77-43d8-a6ea-a559c4ee9fbd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:30.283650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fde67-1c77-43d8-a6ea-a559c4ee9fbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:30.284447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fde67-1c77-43d8-a6ea-a559c4ee9fbd None admin] RBAC: Adding query filter params (interface=public, service_id=8a93831abd314274a373b2a2adb7fb9c, region_id=RegionOne) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:30.284793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fde67-1c77-43d8-a6ea-a559c4ee9fbd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:30.286188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fde67-1c77-43d8-a6ea-a559c4ee9fbd None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:30.292479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 519/1037] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Jul 24 19:42:30 2022] GET /identity/v3/endpoints?service_id=8a93831abd314274a373b2a2adb7fb9c&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:30.298266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fde67-1c77-43d8-a6ea-a559c4ee9fbd None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:30.384185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e952-7f97-4a89-bdaa-cbd962b70859 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:30.393703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:30.431578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fde67-1c77-43d8-a6ea-a559c4ee9fbd None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:30.433743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd431b68-8816-4df3-ae4a-c72d17c760e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:30.434714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd431b68-8816-4df3-ae4a-c72d17c760e8 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:30.435183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd431b68-8816-4df3-ae4a-c72d17c760e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:30.435654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd431b68-8816-4df3-ae4a-c72d17c760e8 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:30.436630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd431b68-8816-4df3-ae4a-c72d17c760e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:30.437108 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd431b68-8816-4df3-ae4a-c72d17c760e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:30.438720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd431b68-8816-4df3-ae4a-c72d17c760e8 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:30.459856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 519/1038] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Jul 24 19:42:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:30.465823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd431b68-8816-4df3-ae4a-c72d17c760e8 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:30.480393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aee350-d527-442e-9f3f-f68352da4be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:30.481121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aee350-d527-442e-9f3f-f68352da4be2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:30.481444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aee350-d527-442e-9f3f-f68352da4be2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:30.481726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aee350-d527-442e-9f3f-f68352da4be2 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:30.482427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aee350-d527-442e-9f3f-f68352da4be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:30.483812 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aee350-d527-442e-9f3f-f68352da4be2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:30.491303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 520/1039] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Jul 24 19:42:30 2022] GET /identity/v3/services => generated 1957 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 19:42:30.595137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd431b68-8816-4df3-ae4a-c72d17c760e8 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:30.637524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8018e-d17a-4848-a361-94c495938b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:30.638100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8018e-d17a-4848-a361-94c495938b6a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:30.638390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8018e-d17a-4848-a361-94c495938b6a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:30.638673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8018e-d17a-4848-a361-94c495938b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:30.681980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:30.739543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8018e-d17a-4848-a361-94c495938b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:30.742366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8018e-d17a-4848-a361-94c495938b6a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:30.864686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a3877-f868-401d-be54-b1452e615876 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:30.864925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a3877-f868-401d-be54-b1452e615876 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:30.865215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a3877-f868-401d-be54-b1452e615876 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:30.866645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 521/1040] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:42:30.872497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6046b-1a6c-4cb5-bd59-b392a8ff5ba1 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:30.872957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6046b-1a6c-4cb5-bd59-b392a8ff5ba1 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:30.873107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6046b-1a6c-4cb5-bd59-b392a8ff5ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:30.891090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-9ff6046b-1a6c-4cb5-bd59-b392a8ff5ba1 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:30.891352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 520/1041] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 24 19:42:30.891968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9ff6046b-1a6c-4cb5-bd59-b392a8ff5ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:30.897667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8018e-d17a-4848-a361-94c495938b6a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:30.928084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:30.975379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 522/1042] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:30.979286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:30.999797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6046b-1a6c-4cb5-bd59-b392a8ff5ba1 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:31.027487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8018e-d17a-4848-a361-94c495938b6a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:31.068858 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42fb9e-0600-41af-9079-0f6790234a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:31.070517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42fb9e-0600-41af-9079-0f6790234a53 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:31.071019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42fb9e-0600-41af-9079-0f6790234a53 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:31.071493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42fb9e-0600-41af-9079-0f6790234a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:31.083653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42fb9e-0600-41af-9079-0f6790234a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:31.085143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42fb9e-0600-41af-9079-0f6790234a53 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-3b42fb9e-0600-41af-9079-0f6790234a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:31.104686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:31.106355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:31.108112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:31.108112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:31.108112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:31.108112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:31.108112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 521/1043] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:31.113368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42fb9e-0600-41af-9079-0f6790234a53 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:31.131719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6046b-1a6c-4cb5-bd59-b392a8ff5ba1 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:31.175874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3906b3-eb8c-4fc8-a89b-dbc8c9eac8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:31.176845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3906b3-eb8c-4fc8-a89b-dbc8c9eac8e7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:31.177365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3906b3-eb8c-4fc8-a89b-dbc8c9eac8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:31.177742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3906b3-eb8c-4fc8-a89b-dbc8c9eac8e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:31.191667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3906b3-eb8c-4fc8-a89b-dbc8c9eac8e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-2a3906b3-eb8c-4fc8-a89b-dbc8c9eac8e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:31.202156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:31.203366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:31.204399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 523/1044] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:31.212670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3906b3-eb8c-4fc8-a89b-dbc8c9eac8e7 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:31.242919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42fb9e-0600-41af-9079-0f6790234a53 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:31.264494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:31.266995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:31.291567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004b212-9e82-4ed6-ae8f-169f5ded34f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:31.292721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004b212-9e82-4ed6-ae8f-169f5ded34f9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:31.293238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004b212-9e82-4ed6-ae8f-169f5ded34f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:31.293732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004b212-9e82-4ed6-ae8f-169f5ded34f9 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:31.294962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004b212-9e82-4ed6-ae8f-169f5ded34f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:31.295522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004b212-9e82-4ed6-ae8f-169f5ded34f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:31.297839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004b212-9e82-4ed6-ae8f-169f5ded34f9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:31.321751 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 522/1045] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:31.323292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004b212-9e82-4ed6-ae8f-169f5ded34f9 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:31.365396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3906b3-eb8c-4fc8-a89b-dbc8c9eac8e7 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:31.412702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34c5b6-e77d-4295-ab2b-3ce4c5334489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:31.413629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c5b6-e77d-4295-ab2b-3ce4c5334489 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:31.414108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c5b6-e77d-4295-ab2b-3ce4c5334489 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:31.414520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c5b6-e77d-4295-ab2b-3ce4c5334489 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:31.415824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34c5b6-e77d-4295-ab2b-3ce4c5334489 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:31.416557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34c5b6-e77d-4295-ab2b-3ce4c5334489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:31.418099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34c5b6-e77d-4295-ab2b-3ce4c5334489 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:31.435256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 524/1046] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:42:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:31.441047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34c5b6-e77d-4295-ab2b-3ce4c5334489 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:31.469225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004b212-9e82-4ed6-ae8f-169f5ded34f9 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:31.513545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7d8bc-d81f-4e2f-a3ae-a6c0912e7b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:31.514773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7d8bc-d81f-4e2f-a3ae-a6c0912e7b6c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:31.515246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7d8bc-d81f-4e2f-a3ae-a6c0912e7b6c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:31.515717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7d8bc-d81f-4e2f-a3ae-a6c0912e7b6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:31.528342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7d8bc-d81f-4e2f-a3ae-a6c0912e7b6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:31.531142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7d8bc-d81f-4e2f-a3ae-a6c0912e7b6c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-25b7d8bc-d81f-4e2f-a3ae-a6c0912e7b6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:31.547876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:31.549809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:31.551553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:31.551553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:31.551553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:31.551553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:31.551553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:31.551553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:31.551553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:31.551553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:31.551553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 523/1047] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:31.555328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7d8bc-d81f-4e2f-a3ae-a6c0912e7b6c None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:31.565179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:31.581307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34c5b6-e77d-4295-ab2b-3ce4c5334489 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:31.595606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:31.633003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a9bb3-1631-448d-89a9-ac01d9f4209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:31.633895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a9bb3-1631-448d-89a9-ac01d9f4209b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:31.634183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a9bb3-1631-448d-89a9-ac01d9f4209b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:31.634541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a9bb3-1631-448d-89a9-ac01d9f4209b None admin] PATH_INFO: `/v3/users/7fce55cb1f41413284d9f50480eb4598` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:31.652704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a9bb3-1631-448d-89a9-ac01d9f4209b None admin] RBAC: Authorizing `identity:get_user(user_id=7fce55cb1f41413284d9f50480eb4598)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:31.652704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a9bb3-1631-448d-89a9-ac01d9f4209b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:31.657697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 525/1048] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:31 2022] GET /identity/v3/users/7fce55cb1f41413284d9f50480eb4598 => generated 285 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:31.658636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a9bb3-1631-448d-89a9-ac01d9f4209b None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:31.721853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7d8bc-d81f-4e2f-a3ae-a6c0912e7b6c None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:31.770056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a9bb3-1631-448d-89a9-ac01d9f4209b None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:31.780625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1425833-0a94-4eb0-9f6f-f561bd93a3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:31.781799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1425833-0a94-4eb0-9f6f-f561bd93a3cb None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:31.782310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1425833-0a94-4eb0-9f6f-f561bd93a3cb None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:31.782806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1425833-0a94-4eb0-9f6f-f561bd93a3cb None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:31.784269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1425833-0a94-4eb0-9f6f-f561bd93a3cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:31.784845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1425833-0a94-4eb0-9f6f-f561bd93a3cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:31.787468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1425833-0a94-4eb0-9f6f-f561bd93a3cb None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:31.806236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478141e9-5b03-4750-8512-41816913688e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:31.806889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478141e9-5b03-4750-8512-41816913688e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:31.807175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478141e9-5b03-4750-8512-41816913688e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:31.807453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478141e9-5b03-4750-8512-41816913688e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:31.815190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 524/1049] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:31.817632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1425833-0a94-4eb0-9f6f-f561bd93a3cb None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:31.830061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:31.912499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478141e9-5b03-4750-8512-41816913688e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:31.914531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478141e9-5b03-4750-8512-41816913688e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:31.941297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:31.956007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1425833-0a94-4eb0-9f6f-f561bd93a3cb None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:32.019178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee693ef4-8798-4e93-8a11-ff4f4e6cb4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:32.020238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee693ef4-8798-4e93-8a11-ff4f4e6cb4f5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:32.020966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee693ef4-8798-4e93-8a11-ff4f4e6cb4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:32.021435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee693ef4-8798-4e93-8a11-ff4f4e6cb4f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:32.024034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee693ef4-8798-4e93-8a11-ff4f4e6cb4f5 None admin] RBAC: Adding query filter params (role.id=39f2429cfacc43b89f5973b368b61cf0, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=123b7f659c6a4ceba18ce268b1335542) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:32.024555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee693ef4-8798-4e93-8a11-ff4f4e6cb4f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:32.027138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee693ef4-8798-4e93-8a11-ff4f4e6cb4f5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:32.040882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 525/1050] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:31 2022] GET /identity/v3/role_assignments?user.id=123b7f659c6a4ceba18ce268b1335542&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=39f2429cfacc43b89f5973b368b61cf0 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:32.079678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 526/1051] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:31 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:42:32.085985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee693ef4-8798-4e93-8a11-ff4f4e6cb4f5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:32.207357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee693ef4-8798-4e93-8a11-ff4f4e6cb4f5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:32.245443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499feb3b-88bd-4033-8c25-8f36e3c7b6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:32.246348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499feb3b-88bd-4033-8c25-8f36e3c7b6a9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:32.246606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499feb3b-88bd-4033-8c25-8f36e3c7b6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:32.246888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499feb3b-88bd-4033-8c25-8f36e3c7b6a9 None admin] PATH_INFO: `/v3/projects/b8edbe7e234a408f94353deca2f08407` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:32.248119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499feb3b-88bd-4033-8c25-8f36e3c7b6a9 None admin] RBAC: Authorizing `identity:get_project(project_id=b8edbe7e234a408f94353deca2f08407)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:32.249609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499feb3b-88bd-4033-8c25-8f36e3c7b6a9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:32.256179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 526/1052] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:42:32 2022] GET /identity/v3/projects/b8edbe7e234a408f94353deca2f08407 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:32.257504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478141e9-5b03-4750-8512-41816913688e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:32.307176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:32.363393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478141e9-5b03-4750-8512-41816913688e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:32.399940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356811c-3dd8-47d1-a778-5fa95d8f66f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:32.400616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356811c-3dd8-47d1-a778-5fa95d8f66f5 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:32.400934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356811c-3dd8-47d1-a778-5fa95d8f66f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:32.401218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356811c-3dd8-47d1-a778-5fa95d8f66f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:32.402060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356811c-3dd8-47d1-a778-5fa95d8f66f5 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=b8edbe7e234a408f94353deca2f08407, user.id=7fce55cb1f41413284d9f50480eb4598) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:32.402379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356811c-3dd8-47d1-a778-5fa95d8f66f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:32.404810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356811c-3dd8-47d1-a778-5fa95d8f66f5 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:32.420757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 527/1053] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:32 2022] GET /identity/v3/role_assignments?user.id=7fce55cb1f41413284d9f50480eb4598&scope.project.id=b8edbe7e234a408f94353deca2f08407&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:32.665423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14808c34-5e4e-44db-8dd0-691e2de0b788 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:32.665620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14808c34-5e4e-44db-8dd0-691e2de0b788 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:32.665908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14808c34-5e4e-44db-8dd0-691e2de0b788 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:32.667599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 527/1054] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:32.673145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da0a75-8b20-4cb5-8839-ebc427914b97 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:32.673444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da0a75-8b20-4cb5-8839-ebc427914b97 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:32.673732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da0a75-8b20-4cb5-8839-ebc427914b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:32.690833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e2da0a75-8b20-4cb5-8839-ebc427914b97 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:32.692035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e2da0a75-8b20-4cb5-8839-ebc427914b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:32.764275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 528/1055] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:32.787751 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14808c34-5e4e-44db-8dd0-691e2de0b788 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:32.820402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:32.899123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14808c34-5e4e-44db-8dd0-691e2de0b788 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:32.937206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023f75f-3af1-42b0-9e5c-d5408faa6f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:32.940000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023f75f-3af1-42b0-9e5c-d5408faa6f59 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:32.940000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023f75f-3af1-42b0-9e5c-d5408faa6f59 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:32.940000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023f75f-3af1-42b0-9e5c-d5408faa6f59 None admin] PATH_INFO: `/v3/services/image` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:32.940000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023f75f-3af1-42b0-9e5c-d5408faa6f59 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:32.940543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023f75f-3af1-42b0-9e5c-d5408faa6f59 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-9023f75f-3af1-42b0-9e5c-d5408faa6f59 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:32.953670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:32.955535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 24 19:42:32.957941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:32.960974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 528/1056] 10.222.0.17 () {56 vars in 1105 bytes} [Sun Jul 24 19:42:32 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:32.966227 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da0a75-8b20-4cb5-8839-ebc427914b97 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:33.035077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:33.134127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da0a75-8b20-4cb5-8839-ebc427914b97 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:33.192574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff8401-670e-4aee-8b76-e07181053854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:33.193642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff8401-670e-4aee-8b76-e07181053854 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:33.194131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff8401-670e-4aee-8b76-e07181053854 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:33.195188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff8401-670e-4aee-8b76-e07181053854 None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:33.196414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff8401-670e-4aee-8b76-e07181053854 None admin] RBAC: Adding query filter params (name=image) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:33.196983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff8401-670e-4aee-8b76-e07181053854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:33.199469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff8401-670e-4aee-8b76-e07181053854 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:33.213196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 529/1057] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:33.214297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023f75f-3af1-42b0-9e5c-d5408faa6f59 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:33.348558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:33.358549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023f75f-3af1-42b0-9e5c-d5408faa6f59 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:33.393594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54010c-3e6e-42f6-9101-7320bdd36ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:33.394274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54010c-3e6e-42f6-9101-7320bdd36ad9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:33.394738 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54010c-3e6e-42f6-9101-7320bdd36ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:33.395028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54010c-3e6e-42f6-9101-7320bdd36ad9 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:33.395745 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54010c-3e6e-42f6-9101-7320bdd36ad9 None admin] RBAC: Adding query filter params (type=image) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:33.396058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54010c-3e6e-42f6-9101-7320bdd36ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:33.397467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54010c-3e6e-42f6-9101-7320bdd36ad9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:33.407783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 529/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:33.409054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff8401-670e-4aee-8b76-e07181053854 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:33.520171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff8401-670e-4aee-8b76-e07181053854 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:33.523137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:33.571379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c597854-4434-4c3b-943d-ca722679bb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:33.572307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c597854-4434-4c3b-943d-ca722679bb99 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:33.572774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c597854-4434-4c3b-943d-ca722679bb99 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:33.573199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c597854-4434-4c3b-943d-ca722679bb99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:33.574143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c597854-4434-4c3b-943d-ca722679bb99 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:33.575668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c597854-4434-4c3b-943d-ca722679bb99 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:33.661138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 530/1059] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Jul 24 19:42:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:34.263632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb605ca-ad10-4a19-8d93-0410eb856ac8 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:34.264039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb605ca-ad10-4a19-8d93-0410eb856ac8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:34.264312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb605ca-ad10-4a19-8d93-0410eb856ac8 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:34.265771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 530/1060] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:34.270941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a6266-284c-4a09-9568-3591b0c68d2d None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:34.271316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a6266-284c-4a09-9568-3591b0c68d2d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:34.271668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a6266-284c-4a09-9568-3591b0c68d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:34.290336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ce7a6266-284c-4a09-9568-3591b0c68d2d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:34.291587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ce7a6266-284c-4a09-9568-3591b0c68d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:34.371861 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 531/1061] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:34.394706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb605ca-ad10-4a19-8d93-0410eb856ac8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:34.486659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9c074-99d2-4190-8d32-3f00dc1f3ad1 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:34.487192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9c074-99d2-4190-8d32-3f00dc1f3ad1 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:34.487279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9c074-99d2-4190-8d32-3f00dc1f3ad1 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:34.488919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 532/1062] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:34.494914 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557e33d-9dc4-4658-a3d4-8a313c0c6177 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:34.495221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557e33d-9dc4-4658-a3d4-8a313c0c6177 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:34.495545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557e33d-9dc4-4658-a3d4-8a313c0c6177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:34.508101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb605ca-ad10-4a19-8d93-0410eb856ac8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:34.512858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e557e33d-9dc4-4658-a3d4-8a313c0c6177 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:34.514122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e557e33d-9dc4-4658-a3d4-8a313c0c6177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:34.542480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c967cd-89e8-4107-8bd3-2cb7237d7522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:34.543295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c967cd-89e8-4107-8bd3-2cb7237d7522 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:34.543579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c967cd-89e8-4107-8bd3-2cb7237d7522 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:34.543853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c967cd-89e8-4107-8bd3-2cb7237d7522 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:34.555460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c967cd-89e8-4107-8bd3-2cb7237d7522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-09c967cd-89e8-4107-8bd3-2cb7237d7522 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:34.563841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:34.564947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:34.565809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 531/1063] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Jul 24 19:42:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:34.571661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c967cd-89e8-4107-8bd3-2cb7237d7522 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:34.589643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 533/1064] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:34.594876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:34.612439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557e33d-9dc4-4658-a3d4-8a313c0c6177 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:34.633293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:34.717337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557e33d-9dc4-4658-a3d4-8a313c0c6177 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:34.737737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c967cd-89e8-4107-8bd3-2cb7237d7522 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:34.754061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ad730-9cbb-44f2-a2ba-9ef0062f662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:34.754869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ad730-9cbb-44f2-a2ba-9ef0062f662f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:34.755160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ad730-9cbb-44f2-a2ba-9ef0062f662f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:34.755448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ad730-9cbb-44f2-a2ba-9ef0062f662f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:34.767484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ad730-9cbb-44f2-a2ba-9ef0062f662f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-8a8ad730-9cbb-44f2-a2ba-9ef0062f662f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:34.775506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:34.776801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:34.777699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 534/1065] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:42:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:34.783053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ad730-9cbb-44f2-a2ba-9ef0062f662f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:34.786299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73048e1d-b844-446e-bdd3-f05e97d39bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:34.786982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73048e1d-b844-446e-bdd3-f05e97d39bf3 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:34.787273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73048e1d-b844-446e-bdd3-f05e97d39bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:34.787560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73048e1d-b844-446e-bdd3-f05e97d39bf3 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:34.788909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73048e1d-b844-446e-bdd3-f05e97d39bf3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:34.789228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73048e1d-b844-446e-bdd3-f05e97d39bf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:34.790606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73048e1d-b844-446e-bdd3-f05e97d39bf3 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:34.804771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 532/1066] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Jul 24 19:42:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:34.807514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73048e1d-b844-446e-bdd3-f05e97d39bf3 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:34.880801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:34.897045 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:34.906321 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ad730-9cbb-44f2-a2ba-9ef0062f662f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:34.921422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73048e1d-b844-446e-bdd3-f05e97d39bf3 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:34.943827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f0355-dfe3-41d9-811c-0f8cc8b96973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:34.944888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f0355-dfe3-41d9-811c-0f8cc8b96973 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:34.945362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f0355-dfe3-41d9-811c-0f8cc8b96973 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:34.945831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f0355-dfe3-41d9-811c-0f8cc8b96973 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:34.946601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f0355-dfe3-41d9-811c-0f8cc8b96973 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:34.947025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f0355-dfe3-41d9-811c-0f8cc8b96973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:34.948280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f0355-dfe3-41d9-811c-0f8cc8b96973 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:34.956767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39c1a3-e2bc-4376-94e5-eb4a8448d0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:34.957353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 535/1067] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:34.957651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39c1a3-e2bc-4376-94e5-eb4a8448d0da None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:34.957946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39c1a3-e2bc-4376-94e5-eb4a8448d0da None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:34.958279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39c1a3-e2bc-4376-94e5-eb4a8448d0da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:34.963008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f0355-dfe3-41d9-811c-0f8cc8b96973 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:34.970086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39c1a3-e2bc-4376-94e5-eb4a8448d0da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:34.971551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39c1a3-e2bc-4376-94e5-eb4a8448d0da None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-5b39c1a3-e2bc-4376-94e5-eb4a8448d0da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:34.989544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:34.990516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:34.992008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:34.992008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:34.992008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:34.992008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:34.992008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 533/1068] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:35.001566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39c1a3-e2bc-4376-94e5-eb4a8448d0da None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:35.087636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f0355-dfe3-41d9-811c-0f8cc8b96973 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:35.117127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39c1a3-e2bc-4376-94e5-eb4a8448d0da None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:35.125769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394436a-7a31-4386-9ca8-88e8bb3f068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:35.126653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394436a-7a31-4386-9ca8-88e8bb3f068d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.126940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394436a-7a31-4386-9ca8-88e8bb3f068d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.127217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394436a-7a31-4386-9ca8-88e8bb3f068d None admin] PATH_INFO: `/v3/users/7fce55cb1f41413284d9f50480eb4598` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.128443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394436a-7a31-4386-9ca8-88e8bb3f068d None admin] RBAC: Authorizing `identity:get_user(user_id=7fce55cb1f41413284d9f50480eb4598)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:35.130516 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394436a-7a31-4386-9ca8-88e8bb3f068d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:35.136932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 536/1069] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:34 2022] GET /identity/v3/users/7fce55cb1f41413284d9f50480eb4598 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:35.137976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394436a-7a31-4386-9ca8-88e8bb3f068d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:35.147074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:35.154375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:35.160702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4403c-026d-4990-abee-963636b85390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:35.161445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4403c-026d-4990-abee-963636b85390 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.161739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4403c-026d-4990-abee-963636b85390 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.162035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4403c-026d-4990-abee-963636b85390 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.162770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4403c-026d-4990-abee-963636b85390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:35.163084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4403c-026d-4990-abee-963636b85390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:35.165279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4403c-026d-4990-abee-963636b85390 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:35.178731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 534/1070] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:35.184143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4403c-026d-4990-abee-963636b85390 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:35.254979 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394436a-7a31-4386-9ca8-88e8bb3f068d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:35.292983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4403c-026d-4990-abee-963636b85390 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:35.294650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac59ef1-07d1-40f0-93b2-69382fb0b44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:35.296116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac59ef1-07d1-40f0-93b2-69382fb0b44e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.296116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac59ef1-07d1-40f0-93b2-69382fb0b44e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.296116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac59ef1-07d1-40f0-93b2-69382fb0b44e None admin] PATH_INFO: `/v3/projects/b8edbe7e234a408f94353deca2f08407` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.297291 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac59ef1-07d1-40f0-93b2-69382fb0b44e None admin] RBAC: Authorizing `identity:get_project(project_id=b8edbe7e234a408f94353deca2f08407)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:35.298735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac59ef1-07d1-40f0-93b2-69382fb0b44e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:35.305379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 537/1071] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:42:35 2022] GET /identity/v3/projects/b8edbe7e234a408f94353deca2f08407 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:35.306365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac59ef1-07d1-40f0-93b2-69382fb0b44e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:35.335889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d54a9e-4b39-494c-bca6-bbfb48fc95d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:35.336724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d54a9e-4b39-494c-bca6-bbfb48fc95d2 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.337018 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d54a9e-4b39-494c-bca6-bbfb48fc95d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.337315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d54a9e-4b39-494c-bca6-bbfb48fc95d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.345265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d54a9e-4b39-494c-bca6-bbfb48fc95d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:35.346658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d54a9e-4b39-494c-bca6-bbfb48fc95d2 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-04d54a9e-4b39-494c-bca6-bbfb48fc95d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:42:35.362447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:35.363639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:42:35.364903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:35.368616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 535/1072] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Jul 24 19:42:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:35.369563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d54a9e-4b39-494c-bca6-bbfb48fc95d2 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:35.394039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:35.429821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:35.431073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac59ef1-07d1-40f0-93b2-69382fb0b44e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:35.470611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081d6b7-ae72-495d-8950-445af909a4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:35.471528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081d6b7-ae72-495d-8950-445af909a4ad None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.471814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081d6b7-ae72-495d-8950-445af909a4ad None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.472139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081d6b7-ae72-495d-8950-445af909a4ad None admin] PATH_INFO: `/v3/projects/b8edbe7e234a408f94353deca2f08407/users/7fce55cb1f41413284d9f50480eb4598/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.474165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081d6b7-ae72-495d-8950-445af909a4ad None admin] RBAC: Authorizing `identity:create_grant(project_id=b8edbe7e234a408f94353deca2f08407, user_id=7fce55cb1f41413284d9f50480eb4598, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:35.475656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081d6b7-ae72-495d-8950-445af909a4ad None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:35.486114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d54a9e-4b39-494c-bca6-bbfb48fc95d2 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:35.524027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef331e2-12a6-4a4f-acee-159fe29777b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:35.524747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef331e2-12a6-4a4f-acee-159fe29777b6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.525037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef331e2-12a6-4a4f-acee-159fe29777b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.525317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef331e2-12a6-4a4f-acee-159fe29777b6 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.526171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef331e2-12a6-4a4f-acee-159fe29777b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:35.526483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef331e2-12a6-4a4f-acee-159fe29777b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:35.527984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef331e2-12a6-4a4f-acee-159fe29777b6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:35.538524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 538/1073] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:42:35 2022] PUT /identity/v3/projects/b8edbe7e234a408f94353deca2f08407/users/7fce55cb1f41413284d9f50480eb4598/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:42:35.549287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 536/1074] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Jul 24 19:42:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:35.555993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081d6b7-ae72-495d-8950-445af909a4ad None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:35.677757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081d6b7-ae72-495d-8950-445af909a4ad None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:35.719520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7bcce-0cb2-4408-9103-a029d530bb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:35.720357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7bcce-0cb2-4408-9103-a029d530bb24 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.720713 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7bcce-0cb2-4408-9103-a029d530bb24 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.720935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7bcce-0cb2-4408-9103-a029d530bb24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.723006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:35.739151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7bcce-0cb2-4408-9103-a029d530bb24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:35.742033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7bcce-0cb2-4408-9103-a029d530bb24 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:35.766357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675e56e-c659-43f0-8e12-39e415818b1f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.766665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675e56e-c659-43f0-8e12-39e415818b1f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.766948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675e56e-c659-43f0-8e12-39e415818b1f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.768306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 537/1075] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:42:35.774239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c74f5c-92a2-4296-8ff7-18572e676d08 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.774531 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c74f5c-92a2-4296-8ff7-18572e676d08 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.774812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c74f5c-92a2-4296-8ff7-18572e676d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-90c7bcce-0cb2-4408-9103-a029d530bb24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:35.776193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:35.777077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:35.777956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:35.777956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:35.777956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:35.777956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:35.778730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 539/1076] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:35.783842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7bcce-0cb2-4408-9103-a029d530bb24 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:35.805097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-09c74f5c-92a2-4296-8ff7-18572e676d08 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:35.806306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-09c74f5c-92a2-4296-8ff7-18572e676d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:35.885822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 538/1077] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:35.907471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7bcce-0cb2-4408-9103-a029d530bb24 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:35.908125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c74f5c-92a2-4296-8ff7-18572e676d08 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:35.948042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bfe70-1893-461a-a763-ea36cc30ddf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:35.949076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bfe70-1893-461a-a763-ea36cc30ddf6 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:35.949830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:35.950407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bfe70-1893-461a-a763-ea36cc30ddf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:35.950845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bfe70-1893-461a-a763-ea36cc30ddf6 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:35.951735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bfe70-1893-461a-a763-ea36cc30ddf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:35.952231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bfe70-1893-461a-a763-ea36cc30ddf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:35.953669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bfe70-1893-461a-a763-ea36cc30ddf6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:35.968058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 540/1078] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:35.973395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bfe70-1893-461a-a763-ea36cc30ddf6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:36.020967 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c74f5c-92a2-4296-8ff7-18572e676d08 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:36.028308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:36.060186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06121d24-0abc-4a69-9bcc-bc8878bbfc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:36.060877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06121d24-0abc-4a69-9bcc-bc8878bbfc10 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:36.061163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06121d24-0abc-4a69-9bcc-bc8878bbfc10 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:36.061437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06121d24-0abc-4a69-9bcc-bc8878bbfc10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:36.088194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bfe70-1893-461a-a763-ea36cc30ddf6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:36.131279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06e6f4-f829-4744-bdf8-0c8aef642be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:36.132116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06e6f4-f829-4744-bdf8-0c8aef642be6 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:36.132400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06e6f4-f829-4744-bdf8-0c8aef642be6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:36.132704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06e6f4-f829-4744-bdf8-0c8aef642be6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:36.137671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06121d24-0abc-4a69-9bcc-bc8878bbfc10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:36.139141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06121d24-0abc-4a69-9bcc-bc8878bbfc10 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:36.140079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06e6f4-f829-4744-bdf8-0c8aef642be6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:36.141686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06e6f4-f829-4744-bdf8-0c8aef642be6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-0b06e6f4-f829-4744-bdf8-0c8aef642be6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:36.157277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:36.158375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:36.159444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:36.159444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:36.159444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:36.159444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:36.159444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:36.159444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:36.159444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:36.159444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:36.167022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 541/1079] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:36.168247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06e6f4-f829-4744-bdf8-0c8aef642be6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:36.183041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:36.256821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 539/1080] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 24 19:42:36.261122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06121d24-0abc-4a69-9bcc-bc8878bbfc10 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:36.289689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06e6f4-f829-4744-bdf8-0c8aef642be6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:36.298761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:36.328246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19236160-3276-41dd-afbc-0af6d5259407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:36.329014 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19236160-3276-41dd-afbc-0af6d5259407 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:36.329311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19236160-3276-41dd-afbc-0af6d5259407 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:36.329592 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19236160-3276-41dd-afbc-0af6d5259407 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:36.330405 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19236160-3276-41dd-afbc-0af6d5259407 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:36.330725 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19236160-3276-41dd-afbc-0af6d5259407 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:36.332888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19236160-3276-41dd-afbc-0af6d5259407 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:36.349526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 542/1081] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:36.355485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19236160-3276-41dd-afbc-0af6d5259407 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:36.411217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06121d24-0abc-4a69-9bcc-bc8878bbfc10 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:36.459455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d80000-3e36-420e-8eb7-a16b484b50a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:36.461113 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d80000-3e36-420e-8eb7-a16b484b50a1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:36.461407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d80000-3e36-420e-8eb7-a16b484b50a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:36.461681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d80000-3e36-420e-8eb7-a16b484b50a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:36.466843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19236160-3276-41dd-afbc-0af6d5259407 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:36.471011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:36.475747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d80000-3e36-420e-8eb7-a16b484b50a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:36.477415 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d80000-3e36-420e-8eb7-a16b484b50a1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-a2d80000-3e36-420e-8eb7-a16b484b50a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:36.501769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:36.503016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:36.503930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:36.503930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:36.503930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:36.503930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:36.503930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 540/1082] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:36.509817 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d1b48-0247-4d1c-a09e-b11f3c586c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:36.509975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d80000-3e36-420e-8eb7-a16b484b50a1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:36.510768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d1b48-0247-4d1c-a09e-b11f3c586c43 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:36.511054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d1b48-0247-4d1c-a09e-b11f3c586c43 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:36.511331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d1b48-0247-4d1c-a09e-b11f3c586c43 None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/123b7f659c6a4ceba18ce268b1335542/roles/39f2429cfacc43b89f5973b368b61cf0` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:36.513711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d1b48-0247-4d1c-a09e-b11f3c586c43 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=123b7f659c6a4ceba18ce268b1335542, role_id=39f2429cfacc43b89f5973b368b61cf0)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:36.515211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d1b48-0247-4d1c-a09e-b11f3c586c43 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:36.584395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 543/1083] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:42:36 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/123b7f659c6a4ceba18ce268b1335542/roles/39f2429cfacc43b89f5973b368b61cf0 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 19:42:36.637249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d80000-3e36-420e-8eb7-a16b484b50a1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:36.680490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8f9b1-a6c0-41a3-9cba-83116010111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:36.681551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f9b1-a6c0-41a3-9cba-83116010111e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:36.681924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f9b1-a6c0-41a3-9cba-83116010111e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:36.682272 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f9b1-a6c0-41a3-9cba-83116010111e None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:36.683154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8f9b1-a6c0-41a3-9cba-83116010111e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:36.683696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8f9b1-a6c0-41a3-9cba-83116010111e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:36.685319 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8f9b1-a6c0-41a3-9cba-83116010111e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:36.700320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 541/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:37.627830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d2af0-9db6-4773-9574-996cccab0d88 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:37.628377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d2af0-9db6-4773-9574-996cccab0d88 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:37.628377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d2af0-9db6-4773-9574-996cccab0d88 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:37.629775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 544/1085] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:37.635635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f08b6-a576-4c5c-a195-5fdce66bc865 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:37.636117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f08b6-a576-4c5c-a195-5fdce66bc865 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:37.636213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f08b6-a576-4c5c-a195-5fdce66bc865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:37.653283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-999f08b6-a576-4c5c-a195-5fdce66bc865 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:37.654455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-999f08b6-a576-4c5c-a195-5fdce66bc865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:37.701904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:37.723443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 542/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:37.747591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d2af0-9db6-4773-9574-996cccab0d88 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:37.774672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:37.849352 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d2af0-9db6-4773-9574-996cccab0d88 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:37.884722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70277a15-8568-433a-9b95-47b145aa5346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:37.885522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70277a15-8568-433a-9b95-47b145aa5346 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:37.885807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70277a15-8568-433a-9b95-47b145aa5346 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:37.886085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70277a15-8568-433a-9b95-47b145aa5346 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:37.896087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70277a15-8568-433a-9b95-47b145aa5346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-70277a15-8568-433a-9b95-47b145aa5346 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:37.903823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:37.905073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:37.905916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 545/1087] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:42:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:37.911343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f08b6-a576-4c5c-a195-5fdce66bc865 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:38.008608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f08b6-a576-4c5c-a195-5fdce66bc865 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:38.042029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fb9d6-3f54-4ec9-a97f-9de13119986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:38.042694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fb9d6-3f54-4ec9-a97f-9de13119986b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:38.042978 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fb9d6-3f54-4ec9-a97f-9de13119986b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:38.043255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fb9d6-3f54-4ec9-a97f-9de13119986b None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:38.044008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fb9d6-3f54-4ec9-a97f-9de13119986b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:38.044314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fb9d6-3f54-4ec9-a97f-9de13119986b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:38.045731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fb9d6-3f54-4ec9-a97f-9de13119986b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:38.054479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 543/1088] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:42:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:38.060367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70277a15-8568-433a-9b95-47b145aa5346 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:38.144340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:38.159502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70277a15-8568-433a-9b95-47b145aa5346 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:38.196565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cf2e7-12cf-4450-88ad-f85b4437609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:38.197479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cf2e7-12cf-4450-88ad-f85b4437609e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:38.197764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cf2e7-12cf-4450-88ad-f85b4437609e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:38.198064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cf2e7-12cf-4450-88ad-f85b4437609e None admin] PATH_INFO: `/v3/users/7fce55cb1f41413284d9f50480eb4598` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:38.199273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cf2e7-12cf-4450-88ad-f85b4437609e None admin] RBAC: Authorizing `identity:get_user(user_id=7fce55cb1f41413284d9f50480eb4598)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:38.200735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cf2e7-12cf-4450-88ad-f85b4437609e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:38.202388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 546/1089] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:38 2022] GET /identity/v3/users/7fce55cb1f41413284d9f50480eb4598 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:38.208091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fb9d6-3f54-4ec9-a97f-9de13119986b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:38.251781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:38.308324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fb9d6-3f54-4ec9-a97f-9de13119986b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:38.346693 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb43b68-e6d6-4a73-861f-ee6c953c75f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:38.347346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb43b68-e6d6-4a73-861f-ee6c953c75f1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:38.347637 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb43b68-e6d6-4a73-861f-ee6c953c75f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:38.347917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb43b68-e6d6-4a73-861f-ee6c953c75f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:38.416207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb43b68-e6d6-4a73-861f-ee6c953c75f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:38.417712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb43b68-e6d6-4a73-861f-ee6c953c75f1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:38.458435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:38.516050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 544/1090] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 24 19:42:38.521882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cf2e7-12cf-4450-88ad-f85b4437609e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:38.625778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cf2e7-12cf-4450-88ad-f85b4437609e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:38.645966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cf358-0bf5-4813-96d7-51fd087e7779 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:38.646517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cf358-0bf5-4813-96d7-51fd087e7779 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:38.646517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cf358-0bf5-4813-96d7-51fd087e7779 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:38.648570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 545/1091] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:42:38.660035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3e5b7-2c84-450a-99cb-31546380631d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:38.660324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3e5b7-2c84-450a-99cb-31546380631d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:38.660599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3e5b7-2c84-450a-99cb-31546380631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:38.677528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e5d01-654c-439b-921f-4bde647e7e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:38.678072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-99b3e5b7-2c84-450a-99cb-31546380631d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:38.678428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e5d01-654c-439b-921f-4bde647e7e36 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:38.678721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e5d01-654c-439b-921f-4bde647e7e36 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:38.678997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e5d01-654c-439b-921f-4bde647e7e36 None admin] PATH_INFO: `/v3/projects/b8edbe7e234a408f94353deca2f08407` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:38.679088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-99b3e5b7-2c84-450a-99cb-31546380631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:38.680902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e5d01-654c-439b-921f-4bde647e7e36 None admin] RBAC: Authorizing `identity:get_project(project_id=b8edbe7e234a408f94353deca2f08407)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:38.682427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e5d01-654c-439b-921f-4bde647e7e36 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:38.684252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 547/1092] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:42:38 2022] GET /identity/v3/projects/b8edbe7e234a408f94353deca2f08407 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:38.690634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e5d01-654c-439b-921f-4bde647e7e36 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:38.720383 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:38.762109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 546/1093] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:42:38.785644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3e5b7-2c84-450a-99cb-31546380631d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:38.807019 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e5d01-654c-439b-921f-4bde647e7e36 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:38.841400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957ca43-69ac-4ffe-9181-894f320aae13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:38.842061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957ca43-69ac-4ffe-9181-894f320aae13 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:38.842343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957ca43-69ac-4ffe-9181-894f320aae13 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:38.842619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957ca43-69ac-4ffe-9181-894f320aae13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:38.843440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957ca43-69ac-4ffe-9181-894f320aae13 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=b8edbe7e234a408f94353deca2f08407, user.id=7fce55cb1f41413284d9f50480eb4598) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:38.843750 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957ca43-69ac-4ffe-9181-894f320aae13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:38.845206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957ca43-69ac-4ffe-9181-894f320aae13 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:38.851792 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 548/1094] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:38 2022] GET /identity/v3/role_assignments?user.id=7fce55cb1f41413284d9f50480eb4598&scope.project.id=b8edbe7e234a408f94353deca2f08407&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:38.859432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:38.895592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3e5b7-2c84-450a-99cb-31546380631d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:38.937629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e8a5d-527d-4df0-b664-8742716fe792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:38.938463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e8a5d-527d-4df0-b664-8742716fe792 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:38.938750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e8a5d-527d-4df0-b664-8742716fe792 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:38.939027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e8a5d-527d-4df0-b664-8742716fe792 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:38.951210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e8a5d-527d-4df0-b664-8742716fe792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-947e8a5d-527d-4df0-b664-8742716fe792 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:38.959600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:38.961406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:38.962259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 547/1095] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Jul 24 19:42:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:38.967682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957ca43-69ac-4ffe-9181-894f320aae13 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:39.057423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:39.072775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957ca43-69ac-4ffe-9181-894f320aae13 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:39.109095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a5572-25a4-450e-9d71-d6340302d51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:39.109775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a5572-25a4-450e-9d71-d6340302d51e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:39.110066 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a5572-25a4-450e-9d71-d6340302d51e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:39.110345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a5572-25a4-450e-9d71-d6340302d51e None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:39.111100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a5572-25a4-450e-9d71-d6340302d51e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:39.111415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a5572-25a4-450e-9d71-d6340302d51e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:39.112805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a5572-25a4-450e-9d71-d6340302d51e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:39.121365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 549/1096] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Jul 24 19:42:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:39.127091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e8a5d-527d-4df0-b664-8742716fe792 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:39.229646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e8a5d-527d-4df0-b664-8742716fe792 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:39.259724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:39.264686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44e057-874e-4c11-b459-9f2395ad3a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:39.265493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44e057-874e-4c11-b459-9f2395ad3a44 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:39.265782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44e057-874e-4c11-b459-9f2395ad3a44 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:39.266064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44e057-874e-4c11-b459-9f2395ad3a44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:39.278567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44e057-874e-4c11-b459-9f2395ad3a44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:39.280072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44e057-874e-4c11-b459-9f2395ad3a44 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-fd44e057-874e-4c11-b459-9f2395ad3a44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:39.299669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:39.300692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:39.301805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:39.301805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:39.301805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:39.301805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:39.301805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 548/1097] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:39.306905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a5572-25a4-450e-9d71-d6340302d51e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:39.407684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a5572-25a4-450e-9d71-d6340302d51e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:39.441775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc5e63-df9e-47bd-a1eb-4ac38b7e6324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:39.442448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc5e63-df9e-47bd-a1eb-4ac38b7e6324 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:39.442735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc5e63-df9e-47bd-a1eb-4ac38b7e6324 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:39.443015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc5e63-df9e-47bd-a1eb-4ac38b7e6324 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:39.443738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc5e63-df9e-47bd-a1eb-4ac38b7e6324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:39.444050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc5e63-df9e-47bd-a1eb-4ac38b7e6324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:39.445986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc5e63-df9e-47bd-a1eb-4ac38b7e6324 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:39.457918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 550/1098] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:39.464169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44e057-874e-4c11-b459-9f2395ad3a44 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:39.563695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44e057-874e-4c11-b459-9f2395ad3a44 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:39.597061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895fdf1-5b28-4565-b2ee-be013b0c572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:39.597861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895fdf1-5b28-4565-b2ee-be013b0c572d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:39.598148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895fdf1-5b28-4565-b2ee-be013b0c572d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:39.598427 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895fdf1-5b28-4565-b2ee-be013b0c572d None admin] PATH_INFO: `/v3/users/nova` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:39.610776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895fdf1-5b28-4565-b2ee-be013b0c572d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:39.612233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895fdf1-5b28-4565-b2ee-be013b0c572d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-9895fdf1-5b28-4565-b2ee-be013b0c572d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:42:39.627433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:39.628416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:39.629736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 549/1099] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Jul 24 19:42:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:39.635002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc5e63-df9e-47bd-a1eb-4ac38b7e6324 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:39.646123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:39.736354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc5e63-df9e-47bd-a1eb-4ac38b7e6324 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:39.774594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b13e1-68eb-4c1f-b62c-cd4e38be9de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:39.775282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b13e1-68eb-4c1f-b62c-cd4e38be9de3 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:39.775572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b13e1-68eb-4c1f-b62c-cd4e38be9de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:39.775854 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b13e1-68eb-4c1f-b62c-cd4e38be9de3 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:39.776744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b13e1-68eb-4c1f-b62c-cd4e38be9de3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:39.777064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b13e1-68eb-4c1f-b62c-cd4e38be9de3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:39.778527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b13e1-68eb-4c1f-b62c-cd4e38be9de3 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:39.802621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 551/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Jul 24 19:42:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:39.804418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895fdf1-5b28-4565-b2ee-be013b0c572d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:39.851899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:39.906474 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895fdf1-5b28-4565-b2ee-be013b0c572d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:39.940196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61429427-902a-4440-ac8f-aace8a61ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:39.940872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61429427-902a-4440-ac8f-aace8a61ff3d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:39.941162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61429427-902a-4440-ac8f-aace8a61ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:39.941440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61429427-902a-4440-ac8f-aace8a61ff3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:40.009546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61429427-902a-4440-ac8f-aace8a61ff3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:40.011045 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61429427-902a-4440-ac8f-aace8a61ff3d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:40.062055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:40.117362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 550/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:42:40.123022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b13e1-68eb-4c1f-b62c-cd4e38be9de3 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:40.188659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:40.221977 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b13e1-68eb-4c1f-b62c-cd4e38be9de3 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:40.256283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bedfb-44b1-4e61-bdfd-b0d153c7962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:40.257129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bedfb-44b1-4e61-bdfd-b0d153c7962a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:40.257416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bedfb-44b1-4e61-bdfd-b0d153c7962a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:40.257690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bedfb-44b1-4e61-bdfd-b0d153c7962a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:40.267826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bedfb-44b1-4e61-bdfd-b0d153c7962a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:40.269329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bedfb-44b1-4e61-bdfd-b0d153c7962a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-df1bedfb-44b1-4e61-bdfd-b0d153c7962a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:42:40.286943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:40.288637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:40.289745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:42:40.289745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:42:40.289745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:42:40.289745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:40.289745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 552/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:40.294864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61429427-902a-4440-ac8f-aace8a61ff3d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:40.393006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61429427-902a-4440-ac8f-aace8a61ff3d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:40.429754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be035dc-aa10-40a9-bd74-da0159d5ed00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:40.430429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be035dc-aa10-40a9-bd74-da0159d5ed00 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:40.430725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be035dc-aa10-40a9-bd74-da0159d5ed00 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:40.431003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be035dc-aa10-40a9-bd74-da0159d5ed00 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:40.431730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be035dc-aa10-40a9-bd74-da0159d5ed00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:40.432042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be035dc-aa10-40a9-bd74-da0159d5ed00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:40.433448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be035dc-aa10-40a9-bd74-da0159d5ed00 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:40.445125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 551/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:40.450658 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bedfb-44b1-4e61-bdfd-b0d153c7962a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:40.550673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bedfb-44b1-4e61-bdfd-b0d153c7962a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:40.567799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:40.583852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c52cc-c996-4776-8de0-a5d77c85c05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:40.587263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c52cc-c996-4776-8de0-a5d77c85c05e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:40.587263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c52cc-c996-4776-8de0-a5d77c85c05e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:40.587263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c52cc-c996-4776-8de0-a5d77c85c05e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:40.592737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c52cc-c996-4776-8de0-a5d77c85c05e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:40.594185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c52cc-c996-4776-8de0-a5d77c85c05e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-a86c52cc-c996-4776-8de0-a5d77c85c05e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:42:40.608763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:40.609717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:40.610673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:42:40.610673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:42:40.610673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:42:40.610673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:42:40.610673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:42:40.610673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:42:40.610673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:42:40.610673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:42:40.610673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 553/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:42:40.616083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be035dc-aa10-40a9-bd74-da0159d5ed00 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:40.620525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:40.718329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be035dc-aa10-40a9-bd74-da0159d5ed00 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:40.751510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdd455-1b91-423c-9e08-64c0234b2884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:40.752173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdd455-1b91-423c-9e08-64c0234b2884 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:40.752459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdd455-1b91-423c-9e08-64c0234b2884 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:40.752774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdd455-1b91-423c-9e08-64c0234b2884 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:40.753564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdd455-1b91-423c-9e08-64c0234b2884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:40.753875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdd455-1b91-423c-9e08-64c0234b2884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:40.755302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdd455-1b91-423c-9e08-64c0234b2884 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:40.768260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 552/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:40.776496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c52cc-c996-4776-8de0-a5d77c85c05e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:40.881239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c52cc-c996-4776-8de0-a5d77c85c05e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:40.915165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9eee6-d80b-4686-aaad-28f2e559ec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:40.915826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9eee6-d80b-4686-aaad-28f2e559ec01 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:40.916362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9eee6-d80b-4686-aaad-28f2e559ec01 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:40.916653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9eee6-d80b-4686-aaad-28f2e559ec01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:40.917491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9eee6-d80b-4686-aaad-28f2e559ec01 None admin] RBAC: Adding query filter params (role.id=39f2429cfacc43b89f5973b368b61cf0, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=123b7f659c6a4ceba18ce268b1335542) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:40.917804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9eee6-d80b-4686-aaad-28f2e559ec01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:40.919227 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9eee6-d80b-4686-aaad-28f2e559ec01 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:40.925138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 554/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:40 2022] GET /identity/v3/role_assignments?user.id=123b7f659c6a4ceba18ce268b1335542&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=39f2429cfacc43b89f5973b368b61cf0 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:40.940677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c25277-df42-4302-ade4-8a1d09d9a6bb None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:40.941268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c25277-df42-4302-ade4-8a1d09d9a6bb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:40.941268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c25277-df42-4302-ade4-8a1d09d9a6bb None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:40.942779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 553/1107] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:40.949194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fea0c1-1408-422e-804c-9d3978b3cc40 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:40.949484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fea0c1-1408-422e-804c-9d3978b3cc40 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:40.949766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fea0c1-1408-422e-804c-9d3978b3cc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:40.968595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d7fea0c1-1408-422e-804c-9d3978b3cc40 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:40.969775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d7fea0c1-1408-422e-804c-9d3978b3cc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:41.010722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:41.052244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 555/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:41.077576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c25277-df42-4302-ade4-8a1d09d9a6bb None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:41.148256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:41.190696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c25277-df42-4302-ade4-8a1d09d9a6bb None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:41.227013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f455976-390e-4620-9397-d5cde4b62b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:41.227824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f455976-390e-4620-9397-d5cde4b62b2d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:41.228111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f455976-390e-4620-9397-d5cde4b62b2d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:41.228389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f455976-390e-4620-9397-d5cde4b62b2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:41.229693 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f455976-390e-4620-9397-d5cde4b62b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:41.231139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f455976-390e-4620-9397-d5cde4b62b2d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:41.232839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 554/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:41.238778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fea0c1-1408-422e-804c-9d3978b3cc40 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:41.346116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fea0c1-1408-422e-804c-9d3978b3cc40 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:41.380913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51239f-0345-4a90-9981-e29e51429830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:41.381595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51239f-0345-4a90-9981-e29e51429830 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:41.381913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51239f-0345-4a90-9981-e29e51429830 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:41.382191 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51239f-0345-4a90-9981-e29e51429830 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:41.382918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51239f-0345-4a90-9981-e29e51429830 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:41.384354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51239f-0345-4a90-9981-e29e51429830 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:41.495768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 556/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:42:41 2022] POST /identity/v3/users => generated 286 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:43.600659 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ece12-acd5-4cb6-beea-eaab78f0cf54 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:43.601204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ece12-acd5-4cb6-beea-eaab78f0cf54 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:43.601204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ece12-acd5-4cb6-beea-eaab78f0cf54 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:43.602601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 555/1111] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:43.607809 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a600c06-412e-4c7e-baa0-55be06e5b557 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:43.608219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a600c06-412e-4c7e-baa0-55be06e5b557 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:43.608362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a600c06-412e-4c7e-baa0-55be06e5b557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:43.626078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-7a600c06-412e-4c7e-baa0-55be06e5b557 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:43.627258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7a600c06-412e-4c7e-baa0-55be06e5b557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:43.650705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:43.696322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 557/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:43.723676 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ece12-acd5-4cb6-beea-eaab78f0cf54 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:43.832930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ece12-acd5-4cb6-beea-eaab78f0cf54 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:43.861304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:43.866928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61de99-71f8-4e88-94da-d782070d77b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:43.867795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61de99-71f8-4e88-94da-d782070d77b5 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:43.868083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61de99-71f8-4e88-94da-d782070d77b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:43.868363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61de99-71f8-4e88-94da-d782070d77b5 None admin] PATH_INFO: `/v3/roles/31a26f4cc3674ca9bdbfc0be860a4318` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:43.869886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61de99-71f8-4e88-94da-d782070d77b5 None admin] RBAC: Authorizing `identity:get_role(role_id=31a26f4cc3674ca9bdbfc0be860a4318)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:43.874974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61de99-71f8-4e88-94da-d782070d77b5 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:43.874974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 556/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:43 2022] GET /identity/v3/roles/31a26f4cc3674ca9bdbfc0be860a4318 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:43.878448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a600c06-412e-4c7e-baa0-55be06e5b557 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:43.974426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a600c06-412e-4c7e-baa0-55be06e5b557 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:44.008329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d3336-2dd0-4bb0-979b-80bfebe6eaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:44.009226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d3336-2dd0-4bb0-979b-80bfebe6eaf0 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:44.009510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d3336-2dd0-4bb0-979b-80bfebe6eaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:44.009785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d3336-2dd0-4bb0-979b-80bfebe6eaf0 None admin] PATH_INFO: `/v3/users/defbb86c367a439cbddedbb622d303f5` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:44.022659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d3336-2dd0-4bb0-979b-80bfebe6eaf0 None admin] RBAC: Authorizing `identity:get_user(user_id=defbb86c367a439cbddedbb622d303f5)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:44.024105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d3336-2dd0-4bb0-979b-80bfebe6eaf0 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:44.025828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 558/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:43 2022] GET /identity/v3/users/defbb86c367a439cbddedbb622d303f5 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:44.031705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61de99-71f8-4e88-94da-d782070d77b5 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:44.138859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61de99-71f8-4e88-94da-d782070d77b5 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:44.173857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd694d0-b0b5-4527-b66a-aaf8256a65d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:44.174508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd694d0-b0b5-4527-b66a-aaf8256a65d1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:44.174797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd694d0-b0b5-4527-b66a-aaf8256a65d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:44.175077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd694d0-b0b5-4527-b66a-aaf8256a65d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:44.238974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:44.241445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd694d0-b0b5-4527-b66a-aaf8256a65d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:44.242904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd694d0-b0b5-4527-b66a-aaf8256a65d1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:44.340759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 557/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:42:44.346626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d3336-2dd0-4bb0-979b-80bfebe6eaf0 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:44.365119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:44.453054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d3336-2dd0-4bb0-979b-80bfebe6eaf0 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:44.490233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d174e1-fa4e-44a2-ac52-ec2699eba808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:44.491057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d174e1-fa4e-44a2-ac52-ec2699eba808 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:44.491343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d174e1-fa4e-44a2-ac52-ec2699eba808 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:44.491626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d174e1-fa4e-44a2-ac52-ec2699eba808 None admin] PATH_INFO: `/v3/projects/b8edbe7e234a408f94353deca2f08407` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:44.492847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d174e1-fa4e-44a2-ac52-ec2699eba808 None admin] RBAC: Authorizing `identity:get_project(project_id=b8edbe7e234a408f94353deca2f08407)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:44.494291 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d174e1-fa4e-44a2-ac52-ec2699eba808 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:44.496020 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 559/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:42:44 2022] GET /identity/v3/projects/b8edbe7e234a408f94353deca2f08407 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:44.502294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd694d0-b0b5-4527-b66a-aaf8256a65d1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:44.597292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:44.600262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd694d0-b0b5-4527-b66a-aaf8256a65d1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:44.633775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcc14e-9c2a-4705-815b-087a8c0de7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:44.634425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc14e-9c2a-4705-815b-087a8c0de7bd None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:44.634707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc14e-9c2a-4705-815b-087a8c0de7bd None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:44.634987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc14e-9c2a-4705-815b-087a8c0de7bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:44.635823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcc14e-9c2a-4705-815b-087a8c0de7bd None admin] RBAC: Adding query filter params (role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=b8edbe7e234a408f94353deca2f08407, user.id=defbb86c367a439cbddedbb622d303f5) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:44.636138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcc14e-9c2a-4705-815b-087a8c0de7bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:44.637602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcc14e-9c2a-4705-815b-087a8c0de7bd None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:44.643153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 558/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:44 2022] GET /identity/v3/role_assignments?user.id=defbb86c367a439cbddedbb622d303f5&scope.project.id=b8edbe7e234a408f94353deca2f08407&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:46.655449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a1be0-a777-4c22-a278-f1477094df55 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:46.655983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a1be0-a777-4c22-a278-f1477094df55 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:46.655983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a1be0-a777-4c22-a278-f1477094df55 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:46.657547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 560/1118] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:46.663318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e9b0e-7793-4094-9531-0a0e67e65d52 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:46.663733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e9b0e-7793-4094-9531-0a0e67e65d52 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:46.663874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e9b0e-7793-4094-9531-0a0e67e65d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:46.680719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e48e9b0e-7793-4094-9531-0a0e67e65d52 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:46.681871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e48e9b0e-7793-4094-9531-0a0e67e65d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:46.751079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 559/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:46.775684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a1be0-a777-4c22-a278-f1477094df55 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:46.871149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:46.889225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a1be0-a777-4c22-a278-f1477094df55 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:46.927131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07ddf3-3ee0-4f92-b8f2-f06ce9c42821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:46.928118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07ddf3-3ee0-4f92-b8f2-f06ce9c42821 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:46.928505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07ddf3-3ee0-4f92-b8f2-f06ce9c42821 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:46.928908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07ddf3-3ee0-4f92-b8f2-f06ce9c42821 None admin] PATH_INFO: `/v3/roles/31a26f4cc3674ca9bdbfc0be860a4318` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:46.930716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07ddf3-3ee0-4f92-b8f2-f06ce9c42821 None admin] RBAC: Authorizing `identity:get_role(role_id=31a26f4cc3674ca9bdbfc0be860a4318)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:46.932526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07ddf3-3ee0-4f92-b8f2-f06ce9c42821 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:46.934080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 561/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:46 2022] GET /identity/v3/roles/31a26f4cc3674ca9bdbfc0be860a4318 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:46.939845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e9b0e-7793-4094-9531-0a0e67e65d52 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:47.031646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:47.038117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e9b0e-7793-4094-9531-0a0e67e65d52 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:47.072445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8ede4-ce92-466d-a3d9-2c9e0f11bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:47.073351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8ede4-ce92-466d-a3d9-2c9e0f11bf30 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:47.073636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8ede4-ce92-466d-a3d9-2c9e0f11bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:47.073914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8ede4-ce92-466d-a3d9-2c9e0f11bf30 None admin] PATH_INFO: `/v3/users/defbb86c367a439cbddedbb622d303f5` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:47.075169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8ede4-ce92-466d-a3d9-2c9e0f11bf30 None admin] RBAC: Authorizing `identity:get_user(user_id=defbb86c367a439cbddedbb622d303f5)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:47.076619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8ede4-ce92-466d-a3d9-2c9e0f11bf30 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:47.078328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 560/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:46 2022] GET /identity/v3/users/defbb86c367a439cbddedbb622d303f5 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:47.085180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07ddf3-3ee0-4f92-b8f2-f06ce9c42821 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:47.197757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07ddf3-3ee0-4f92-b8f2-f06ce9c42821 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:47.235874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030c86e-07b3-45fb-8bb2-680e2ce29287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:47.236811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030c86e-07b3-45fb-8bb2-680e2ce29287 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:47.237095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030c86e-07b3-45fb-8bb2-680e2ce29287 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:47.237373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030c86e-07b3-45fb-8bb2-680e2ce29287 None admin] PATH_INFO: `/v3/projects/b8edbe7e234a408f94353deca2f08407` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:47.238545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030c86e-07b3-45fb-8bb2-680e2ce29287 None admin] RBAC: Authorizing `identity:get_project(project_id=b8edbe7e234a408f94353deca2f08407)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:47.240104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030c86e-07b3-45fb-8bb2-680e2ce29287 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:47.242339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 562/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:42:47 2022] GET /identity/v3/projects/b8edbe7e234a408f94353deca2f08407 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:47.248456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8ede4-ce92-466d-a3d9-2c9e0f11bf30 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:47.347123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8ede4-ce92-466d-a3d9-2c9e0f11bf30 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:47.383651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b61b22-26e6-4e71-84dc-93017c3b30db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:47.384568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b61b22-26e6-4e71-84dc-93017c3b30db None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:47.384889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b61b22-26e6-4e71-84dc-93017c3b30db None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:47.385173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b61b22-26e6-4e71-84dc-93017c3b30db None admin] PATH_INFO: `/v3/projects/b8edbe7e234a408f94353deca2f08407/users/defbb86c367a439cbddedbb622d303f5/roles/31a26f4cc3674ca9bdbfc0be860a4318` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:47.387444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b61b22-26e6-4e71-84dc-93017c3b30db None admin] RBAC: Authorizing `identity:create_grant(project_id=b8edbe7e234a408f94353deca2f08407, user_id=defbb86c367a439cbddedbb622d303f5, role_id=31a26f4cc3674ca9bdbfc0be860a4318)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:47.388937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b61b22-26e6-4e71-84dc-93017c3b30db None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:47.529835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 561/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:42:47 2022] PUT /identity/v3/projects/b8edbe7e234a408f94353deca2f08407/users/defbb86c367a439cbddedbb622d303f5/roles/31a26f4cc3674ca9bdbfc0be860a4318 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:42:49.577904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818d041-4493-46f2-9636-268005edc948 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:49.578475 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818d041-4493-46f2-9636-268005edc948 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:49.578613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818d041-4493-46f2-9636-268005edc948 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:49.580005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 563/1124] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:49.587766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617094a-864d-4a99-bd57-ad4c36ffa29d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:49.588366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617094a-864d-4a99-bd57-ad4c36ffa29d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:49.588366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617094a-864d-4a99-bd57-ad4c36ffa29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:49.609165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-0617094a-864d-4a99-bd57-ad4c36ffa29d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:49.610373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0617094a-864d-4a99-bd57-ad4c36ffa29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:49.628479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:49.681522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 562/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:49.705482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818d041-4493-46f2-9636-268005edc948 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:49.734641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:49.833732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818d041-4493-46f2-9636-268005edc948 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:49.871136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326af4a4-04de-4c64-b0fd-2824c70da70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:49.872011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326af4a4-04de-4c64-b0fd-2824c70da70e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:49.872296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326af4a4-04de-4c64-b0fd-2824c70da70e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:49.872575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326af4a4-04de-4c64-b0fd-2824c70da70e None admin] PATH_INFO: `/v3/roles/31a26f4cc3674ca9bdbfc0be860a4318` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:49.874120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326af4a4-04de-4c64-b0fd-2824c70da70e None admin] RBAC: Authorizing `identity:get_role(role_id=31a26f4cc3674ca9bdbfc0be860a4318)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:49.875495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326af4a4-04de-4c64-b0fd-2824c70da70e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:49.876942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 564/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:49 2022] GET /identity/v3/roles/31a26f4cc3674ca9bdbfc0be860a4318 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:49.883608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617094a-864d-4a99-bd57-ad4c36ffa29d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:49.998702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617094a-864d-4a99-bd57-ad4c36ffa29d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:50.038406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e343-2fbf-4fda-9d8d-6980a8a4e106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:50.039274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e343-2fbf-4fda-9d8d-6980a8a4e106 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:50.039556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e343-2fbf-4fda-9d8d-6980a8a4e106 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:50.039828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e343-2fbf-4fda-9d8d-6980a8a4e106 None admin] PATH_INFO: `/v3/users/defbb86c367a439cbddedbb622d303f5` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:50.041304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9e343-2fbf-4fda-9d8d-6980a8a4e106 None admin] RBAC: Authorizing `identity:get_user(user_id=defbb86c367a439cbddedbb622d303f5)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:50.042726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9e343-2fbf-4fda-9d8d-6980a8a4e106 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:50.044374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 563/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:49 2022] GET /identity/v3/users/defbb86c367a439cbddedbb622d303f5 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:50.050326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326af4a4-04de-4c64-b0fd-2824c70da70e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:50.144130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:50.157471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326af4a4-04de-4c64-b0fd-2824c70da70e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:50.196202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f1d2d-8176-45d8-b51d-1e34260ddf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:50.196900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1d2d-8176-45d8-b51d-1e34260ddf2a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:50.197185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1d2d-8176-45d8-b51d-1e34260ddf2a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:50.197464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1d2d-8176-45d8-b51d-1e34260ddf2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:50.270955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f1d2d-8176-45d8-b51d-1e34260ddf2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:50.270955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f1d2d-8176-45d8-b51d-1e34260ddf2a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:50.360952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:50.374864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 565/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:42:50.381256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e343-2fbf-4fda-9d8d-6980a8a4e106 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:50.410215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:50.520410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e343-2fbf-4fda-9d8d-6980a8a4e106 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:50.561138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd73f4-4548-428b-b584-629e8c0cf9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:50.562182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd73f4-4548-428b-b584-629e8c0cf9f4 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:50.562521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd73f4-4548-428b-b584-629e8c0cf9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:50.562841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd73f4-4548-428b-b584-629e8c0cf9f4 None admin] PATH_INFO: `/v3/projects/b8edbe7e234a408f94353deca2f08407` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:50.564129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd73f4-4548-428b-b584-629e8c0cf9f4 None admin] RBAC: Authorizing `identity:get_project(project_id=b8edbe7e234a408f94353deca2f08407)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:50.565671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd73f4-4548-428b-b584-629e8c0cf9f4 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:50.567467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 564/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:42:50 2022] GET /identity/v3/projects/b8edbe7e234a408f94353deca2f08407 => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:50.573980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f1d2d-8176-45d8-b51d-1e34260ddf2a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:50.680938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f1d2d-8176-45d8-b51d-1e34260ddf2a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:50.715297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264836d2-3929-4479-a5e5-29c4f2b123ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:50.715950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264836d2-3929-4479-a5e5-29c4f2b123ac None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:50.716233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264836d2-3929-4479-a5e5-29c4f2b123ac None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:50.716507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264836d2-3929-4479-a5e5-29c4f2b123ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:50.717427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264836d2-3929-4479-a5e5-29c4f2b123ac None admin] RBAC: Adding query filter params (role.id=31a26f4cc3674ca9bdbfc0be860a4318, scope.project.id=b8edbe7e234a408f94353deca2f08407, user.id=defbb86c367a439cbddedbb622d303f5) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:50.717735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264836d2-3929-4479-a5e5-29c4f2b123ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:50.719197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264836d2-3929-4479-a5e5-29c4f2b123ac None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:50.726563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 566/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:50 2022] GET /identity/v3/role_assignments?user.id=defbb86c367a439cbddedbb622d303f5&scope.project.id=b8edbe7e234a408f94353deca2f08407&role.id=31a26f4cc3674ca9bdbfc0be860a4318 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:42:52.789684 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db25187-b674-4de5-ada5-677299c9670f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:52.790205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db25187-b674-4de5-ada5-677299c9670f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:52.790299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db25187-b674-4de5-ada5-677299c9670f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:52.791879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 565/1131] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:52.797181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a8d17-4a61-4b72-a7ed-97215c744a4a None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:52.797622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a8d17-4a61-4b72-a7ed-97215c744a4a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:52.797835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a8d17-4a61-4b72-a7ed-97215c744a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:52.814901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-483a8d17-4a61-4b72-a7ed-97215c744a4a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:52.816096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-483a8d17-4a61-4b72-a7ed-97215c744a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:52.862553 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:52.882759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 567/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:52.905941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db25187-b674-4de5-ada5-677299c9670f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:52.993265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:53.005579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db25187-b674-4de5-ada5-677299c9670f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:53.038631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a2d2b-38bf-4fcb-81d3-4bb08da3303e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:53.039414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a2d2b-38bf-4fcb-81d3-4bb08da3303e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:53.039746 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a2d2b-38bf-4fcb-81d3-4bb08da3303e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:53.039970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a2d2b-38bf-4fcb-81d3-4bb08da3303e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:53.041402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a2d2b-38bf-4fcb-81d3-4bb08da3303e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:53.042825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a2d2b-38bf-4fcb-81d3-4bb08da3303e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:53.044444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 566/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:53.050418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a8d17-4a61-4b72-a7ed-97215c744a4a None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:53.149098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a8d17-4a61-4b72-a7ed-97215c744a4a None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:53.182930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf7292-6eb3-4617-9f2c-172bbfdba092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:53.183586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf7292-6eb3-4617-9f2c-172bbfdba092 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:53.183868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf7292-6eb3-4617-9f2c-172bbfdba092 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:53.184138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf7292-6eb3-4617-9f2c-172bbfdba092 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:53.184869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf7292-6eb3-4617-9f2c-172bbfdba092 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:53.186290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf7292-6eb3-4617-9f2c-172bbfdba092 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:53.270224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 568/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:42:53 2022] POST /identity/v3/projects => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:55.420945 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48263e0d-8c74-4bf4-b7c4-fbb1452ee140 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:55.421634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48263e0d-8c74-4bf4-b7c4-fbb1452ee140 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:55.421634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48263e0d-8c74-4bf4-b7c4-fbb1452ee140 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:55.422869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 567/1135] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:55.428406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c541100-12c3-4e33-ad5d-0b563ece427f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:55.428850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c541100-12c3-4e33-ad5d-0b563ece427f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:55.429017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c541100-12c3-4e33-ad5d-0b563ece427f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:55.446208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-0c541100-12c3-4e33-ad5d-0b563ece427f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:55.447396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0c541100-12c3-4e33-ad5d-0b563ece427f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:55.474023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:55.514017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 569/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:55.539489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48263e0d-8c74-4bf4-b7c4-fbb1452ee140 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:55.644404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48263e0d-8c74-4bf4-b7c4-fbb1452ee140 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:55.679830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0590b59-3e35-4409-ae89-af667a995ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:55.680548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0590b59-3e35-4409-ae89-af667a995ae9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:55.680877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0590b59-3e35-4409-ae89-af667a995ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:55.681151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0590b59-3e35-4409-ae89-af667a995ae9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:55.682336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0590b59-3e35-4409-ae89-af667a995ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:55.683770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0590b59-3e35-4409-ae89-af667a995ae9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:55.685446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 568/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:42:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:55.691381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c541100-12c3-4e33-ad5d-0b563ece427f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:55.794886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c541100-12c3-4e33-ad5d-0b563ece427f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:55.827874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6beec-008d-4f28-894a-7b31434ca647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:55.828539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6beec-008d-4f28-894a-7b31434ca647 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:55.828847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6beec-008d-4f28-894a-7b31434ca647 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:55.829124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6beec-008d-4f28-894a-7b31434ca647 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:55.829835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6beec-008d-4f28-894a-7b31434ca647 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:55.831248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6beec-008d-4f28-894a-7b31434ca647 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:55.897453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 570/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:42:55 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:42:57.989107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83286b03-d72d-45ce-a341-d3b01d739ceb None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:57.989661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83286b03-d72d-45ce-a341-d3b01d739ceb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:57.989661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83286b03-d72d-45ce-a341-d3b01d739ceb None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:57.991600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 569/1139] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:42:57.996948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f196e1-5389-437a-9bf0-b6f2d5f2c496 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:57.997373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f196e1-5389-437a-9bf0-b6f2d5f2c496 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:57.997524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f196e1-5389-437a-9bf0-b6f2d5f2c496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:58.014114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a1f196e1-5389-437a-9bf0-b6f2d5f2c496 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:42:58.015303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a1f196e1-5389-437a-9bf0-b6f2d5f2c496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:42:58.029154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:58.081425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 571/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:42:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:42:58.105115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83286b03-d72d-45ce-a341-d3b01d739ceb None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:58.130377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:58.205246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83286b03-d72d-45ce-a341-d3b01d739ceb None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:58.237925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94593aca-144a-4151-aee3-42a505d01e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:58.238707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94593aca-144a-4151-aee3-42a505d01e17 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:58.238995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94593aca-144a-4151-aee3-42a505d01e17 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:58.239324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94593aca-144a-4151-aee3-42a505d01e17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:58.249706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94593aca-144a-4151-aee3-42a505d01e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-94593aca-144a-4151-aee3-42a505d01e17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:42:58.257379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:42:58.258483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:42:58.259355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 570/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:42:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:42:58.264599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f196e1-5389-437a-9bf0-b6f2d5f2c496 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:58.363425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f196e1-5389-437a-9bf0-b6f2d5f2c496 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:58.400033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cc394-1af0-4040-a560-fc5af7034e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:58.400744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cc394-1af0-4040-a560-fc5af7034e4e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:58.401034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cc394-1af0-4040-a560-fc5af7034e4e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:58.401310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cc394-1af0-4040-a560-fc5af7034e4e None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:58.402073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cc394-1af0-4040-a560-fc5af7034e4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:58.402385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cc394-1af0-4040-a560-fc5af7034e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:58.403829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cc394-1af0-4040-a560-fc5af7034e4e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:58.413710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 572/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:42:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:58.419652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94593aca-144a-4151-aee3-42a505d01e17 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:58.501387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:58.516204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94593aca-144a-4151-aee3-42a505d01e17 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:58.551133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb8c78-a7d2-4001-a7ae-589a6dcc98f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:58.551992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb8c78-a7d2-4001-a7ae-589a6dcc98f7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:58.552272 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb8c78-a7d2-4001-a7ae-589a6dcc98f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:58.552545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb8c78-a7d2-4001-a7ae-589a6dcc98f7 None admin] PATH_INFO: `/v3/users/95b14db19be84d7e98fd0035da07c9be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:58.565146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb8c78-a7d2-4001-a7ae-589a6dcc98f7 None admin] RBAC: Authorizing `identity:get_user(user_id=95b14db19be84d7e98fd0035da07c9be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:58.566566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb8c78-a7d2-4001-a7ae-589a6dcc98f7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:58.568255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 571/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:42:58 2022] GET /identity/v3/users/95b14db19be84d7e98fd0035da07c9be => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:58.574172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cc394-1af0-4040-a560-fc5af7034e4e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:58.589795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:58.673528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cc394-1af0-4040-a560-fc5af7034e4e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:58.705842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61d2a2-a03d-4a62-898b-1a7a1edb4f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:58.706487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61d2a2-a03d-4a62-898b-1a7a1edb4f3d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:58.706772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61d2a2-a03d-4a62-898b-1a7a1edb4f3d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:58.707048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61d2a2-a03d-4a62-898b-1a7a1edb4f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:58.778759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61d2a2-a03d-4a62-898b-1a7a1edb4f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:58.780118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61d2a2-a03d-4a62-898b-1a7a1edb4f3d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:58.793559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:58.876359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 573/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:42:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:42:58.882943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb8c78-a7d2-4001-a7ae-589a6dcc98f7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:58.984293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb8c78-a7d2-4001-a7ae-589a6dcc98f7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:59.016764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0fd0c-6849-4917-9b18-09e7950d5654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:59.017616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0fd0c-6849-4917-9b18-09e7950d5654 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:59.017898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0fd0c-6849-4917-9b18-09e7950d5654 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:59.018172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0fd0c-6849-4917-9b18-09e7950d5654 None admin] PATH_INFO: `/v3/projects/0447c7029ab74abdbd90d7041e4982b6` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:59.019333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0fd0c-6849-4917-9b18-09e7950d5654 None admin] RBAC: Authorizing `identity:get_project(project_id=0447c7029ab74abdbd90d7041e4982b6)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:59.020777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0fd0c-6849-4917-9b18-09e7950d5654 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:59.023125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 572/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:42:58 2022] GET /identity/v3/projects/0447c7029ab74abdbd90d7041e4982b6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:42:59.029439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61d2a2-a03d-4a62-898b-1a7a1edb4f3d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:42:59.125204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61d2a2-a03d-4a62-898b-1a7a1edb4f3d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:42:59.137896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:42:59.158495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff045b2e-93d6-4305-adcc-dad3f59de81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:42:59.159145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff045b2e-93d6-4305-adcc-dad3f59de81b None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:42:59.159430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff045b2e-93d6-4305-adcc-dad3f59de81b None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:42:59.159704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff045b2e-93d6-4305-adcc-dad3f59de81b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:42:59.160523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff045b2e-93d6-4305-adcc-dad3f59de81b None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=0447c7029ab74abdbd90d7041e4982b6, user.id=95b14db19be84d7e98fd0035da07c9be) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:42:59.160868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff045b2e-93d6-4305-adcc-dad3f59de81b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:42:59.162846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff045b2e-93d6-4305-adcc-dad3f59de81b None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:42:59.168509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 574/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:42:59 2022] GET /identity/v3/role_assignments?user.id=95b14db19be84d7e98fd0035da07c9be&scope.project.id=0447c7029ab74abdbd90d7041e4982b6&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:01.203117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8c9af-85e8-4d63-ad2f-3487921b71bb None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:01.203687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8c9af-85e8-4d63-ad2f-3487921b71bb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:01.203785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8c9af-85e8-4d63-ad2f-3487921b71bb None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:01.205233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 573/1147] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:43:01.211244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fdacd-3ada-400e-84de-145ab52a8aa5 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:01.211686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fdacd-3ada-400e-84de-145ab52a8aa5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:01.211803 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fdacd-3ada-400e-84de-145ab52a8aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:01.228292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-5f2fdacd-3ada-400e-84de-145ab52a8aa5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:43:01.229506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5f2fdacd-3ada-400e-84de-145ab52a8aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:43:01.294660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 575/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:43:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:43:01.317077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8c9af-85e8-4d63-ad2f-3487921b71bb None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:01.330311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:01.423046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8c9af-85e8-4d63-ad2f-3487921b71bb None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:01.457818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a06488-7f6a-4a34-9ea7-e4d1946ab14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:01.458535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a06488-7f6a-4a34-9ea7-e4d1946ab14d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:01.458856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a06488-7f6a-4a34-9ea7-e4d1946ab14d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:01.459137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a06488-7f6a-4a34-9ea7-e4d1946ab14d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:01.469348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a06488-7f6a-4a34-9ea7-e4d1946ab14d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-90a06488-7f6a-4a34-9ea7-e4d1946ab14d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:43:01.477172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:43:01.478343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:43:01.479478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 574/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:43:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:43:01.484745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fdacd-3ada-400e-84de-145ab52a8aa5 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:01.581360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fdacd-3ada-400e-84de-145ab52a8aa5 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:01.588533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:01.613988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395083b3-7dad-4eae-8b56-2bf308bda5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:01.614652 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395083b3-7dad-4eae-8b56-2bf308bda5f4 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:01.615445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395083b3-7dad-4eae-8b56-2bf308bda5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:01.615735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395083b3-7dad-4eae-8b56-2bf308bda5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:01.616473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395083b3-7dad-4eae-8b56-2bf308bda5f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:43:01.616812 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395083b3-7dad-4eae-8b56-2bf308bda5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:01.618211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395083b3-7dad-4eae-8b56-2bf308bda5f4 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:01.626434 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 576/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:43:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:01.633234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a06488-7f6a-4a34-9ea7-e4d1946ab14d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:01.700971 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:01.731236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a06488-7f6a-4a34-9ea7-e4d1946ab14d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:01.765274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8ea49-0d68-4212-b4dc-09294626e515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:01.766140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8ea49-0d68-4212-b4dc-09294626e515 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:01.766428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8ea49-0d68-4212-b4dc-09294626e515 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:01.766704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8ea49-0d68-4212-b4dc-09294626e515 None admin] PATH_INFO: `/v3/users/95b14db19be84d7e98fd0035da07c9be` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:01.767886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8ea49-0d68-4212-b4dc-09294626e515 None admin] RBAC: Authorizing `identity:get_user(user_id=95b14db19be84d7e98fd0035da07c9be)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:01.769347 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8ea49-0d68-4212-b4dc-09294626e515 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:01.770994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 575/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:43:01 2022] GET /identity/v3/users/95b14db19be84d7e98fd0035da07c9be => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:01.777080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395083b3-7dad-4eae-8b56-2bf308bda5f4 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:01.881458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395083b3-7dad-4eae-8b56-2bf308bda5f4 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:01.913630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498bef5-5415-4444-a70c-a392fc52dc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:01.914474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498bef5-5415-4444-a70c-a392fc52dc2e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:01.914756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498bef5-5415-4444-a70c-a392fc52dc2e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:01.915030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498bef5-5415-4444-a70c-a392fc52dc2e None admin] PATH_INFO: `/v3/projects/0447c7029ab74abdbd90d7041e4982b6` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:01.916452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498bef5-5415-4444-a70c-a392fc52dc2e None admin] RBAC: Authorizing `identity:get_project(project_id=0447c7029ab74abdbd90d7041e4982b6)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:01.917913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498bef5-5415-4444-a70c-a392fc52dc2e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:01.919604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 577/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:43:01 2022] GET /identity/v3/projects/0447c7029ab74abdbd90d7041e4982b6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:01.925582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8ea49-0d68-4212-b4dc-09294626e515 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:02.061661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8ea49-0d68-4212-b4dc-09294626e515 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:02.094158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b514ab-5f91-41a3-ae65-3f2c9c9a26e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:02.095203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b514ab-5f91-41a3-ae65-3f2c9c9a26e1 None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:02.095489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b514ab-5f91-41a3-ae65-3f2c9c9a26e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:02.095768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b514ab-5f91-41a3-ae65-3f2c9c9a26e1 None admin] PATH_INFO: `/v3/projects/0447c7029ab74abdbd90d7041e4982b6/users/95b14db19be84d7e98fd0035da07c9be/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:02.097780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b514ab-5f91-41a3-ae65-3f2c9c9a26e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0447c7029ab74abdbd90d7041e4982b6, user_id=95b14db19be84d7e98fd0035da07c9be, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:02.099217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b514ab-5f91-41a3-ae65-3f2c9c9a26e1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:02.191542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:02.197220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 576/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:43:01 2022] PUT /identity/v3/projects/0447c7029ab74abdbd90d7041e4982b6/users/95b14db19be84d7e98fd0035da07c9be/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:43:04.232969 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e2e88-44f8-4a03-a83b-5139bb66ca55 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:04.233518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e2e88-44f8-4a03-a83b-5139bb66ca55 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:04.233518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e2e88-44f8-4a03-a83b-5139bb66ca55 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:04.234894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 578/1154] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:43:04.240754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc824f1-aae7-424e-8eb4-09beb247568b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:04.241140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc824f1-aae7-424e-8eb4-09beb247568b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:04.241330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc824f1-aae7-424e-8eb4-09beb247568b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:04.258133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-dbc824f1-aae7-424e-8eb4-09beb247568b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:43:04.259348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-dbc824f1-aae7-424e-8eb4-09beb247568b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:43:04.334639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 577/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:43:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:43:04.358448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e2e88-44f8-4a03-a83b-5139bb66ca55 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:04.395411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:04.461624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e2e88-44f8-4a03-a83b-5139bb66ca55 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:04.495049 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3f731-ccab-48cb-8ab9-7a99e2429de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:04.495845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3f731-ccab-48cb-8ab9-7a99e2429de9 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:04.496134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3f731-ccab-48cb-8ab9-7a99e2429de9 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:04.496419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3f731-ccab-48cb-8ab9-7a99e2429de9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:04.507024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3f731-ccab-48cb-8ab9-7a99e2429de9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-dbd3f731-ccab-48cb-8ab9-7a99e2429de9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:43:04.514942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:43:04.515995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:43:04.516947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 579/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:43:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:43:04.522729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc824f1-aae7-424e-8eb4-09beb247568b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:04.625549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc824f1-aae7-424e-8eb4-09beb247568b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:04.656328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:04.666334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64b27f-5833-4cf7-81a0-5587aa7f2341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:04.667007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64b27f-5833-4cf7-81a0-5587aa7f2341 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:04.673107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64b27f-5833-4cf7-81a0-5587aa7f2341 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:04.673107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64b27f-5833-4cf7-81a0-5587aa7f2341 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:04.673107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64b27f-5833-4cf7-81a0-5587aa7f2341 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:43:04.673107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64b27f-5833-4cf7-81a0-5587aa7f2341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:04.673107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64b27f-5833-4cf7-81a0-5587aa7f2341 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:04.681009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 578/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:43:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:04.686630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3f731-ccab-48cb-8ab9-7a99e2429de9 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:04.782893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:04.788576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3f731-ccab-48cb-8ab9-7a99e2429de9 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:04.822219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cefef1-0e15-44dc-b403-00c73552f0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:04.823088 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cefef1-0e15-44dc-b403-00c73552f0cf None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:04.823374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cefef1-0e15-44dc-b403-00c73552f0cf None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:04.823654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cefef1-0e15-44dc-b403-00c73552f0cf None admin] PATH_INFO: `/v3/users/95b14db19be84d7e98fd0035da07c9be` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:04.824861 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cefef1-0e15-44dc-b403-00c73552f0cf None admin] RBAC: Authorizing `identity:get_user(user_id=95b14db19be84d7e98fd0035da07c9be)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:04.826295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cefef1-0e15-44dc-b403-00c73552f0cf None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:04.827932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 580/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:43:04 2022] GET /identity/v3/users/95b14db19be84d7e98fd0035da07c9be => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:04.834419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64b27f-5833-4cf7-81a0-5587aa7f2341 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:04.936786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64b27f-5833-4cf7-81a0-5587aa7f2341 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:04.986645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2a827-3aac-4591-8f74-01a0aa614ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:04.987294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2a827-3aac-4591-8f74-01a0aa614ca2 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:04.987581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2a827-3aac-4591-8f74-01a0aa614ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:04.987857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2a827-3aac-4591-8f74-01a0aa614ca2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:05.044044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:05.053882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2a827-3aac-4591-8f74-01a0aa614ca2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:05.055346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2a827-3aac-4591-8f74-01a0aa614ca2 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:05.152937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 579/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:43:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:43:05.158962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cefef1-0e15-44dc-b403-00c73552f0cf None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:05.256524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cefef1-0e15-44dc-b403-00c73552f0cf None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:05.289964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24dd4f-99eb-4374-8675-e9b4f49b7ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:05.291070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24dd4f-99eb-4374-8675-e9b4f49b7ac1 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:05.291350 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24dd4f-99eb-4374-8675-e9b4f49b7ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:05.291622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24dd4f-99eb-4374-8675-e9b4f49b7ac1 None admin] PATH_INFO: `/v3/projects/0447c7029ab74abdbd90d7041e4982b6` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:05.292818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24dd4f-99eb-4374-8675-e9b4f49b7ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=0447c7029ab74abdbd90d7041e4982b6)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:05.294292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24dd4f-99eb-4374-8675-e9b4f49b7ac1 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:05.295921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 581/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:43:05 2022] GET /identity/v3/projects/0447c7029ab74abdbd90d7041e4982b6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:05.301979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2a827-3aac-4591-8f74-01a0aa614ca2 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:05.388463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:05.397661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2a827-3aac-4591-8f74-01a0aa614ca2 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:05.431421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d5ba4-f5ec-4a38-8be4-7d8dfcfcc721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:05.432087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d5ba4-f5ec-4a38-8be4-7d8dfcfcc721 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:05.432374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d5ba4-f5ec-4a38-8be4-7d8dfcfcc721 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:05.432661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d5ba4-f5ec-4a38-8be4-7d8dfcfcc721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:05.433513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d5ba4-f5ec-4a38-8be4-7d8dfcfcc721 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=0447c7029ab74abdbd90d7041e4982b6, user.id=95b14db19be84d7e98fd0035da07c9be) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:43:05.433830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d5ba4-f5ec-4a38-8be4-7d8dfcfcc721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:05.435258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d5ba4-f5ec-4a38-8be4-7d8dfcfcc721 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:05.441158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 580/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:43:05 2022] GET /identity/v3/role_assignments?user.id=95b14db19be84d7e98fd0035da07c9be&scope.project.id=0447c7029ab74abdbd90d7041e4982b6&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:07.488693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390648b-ad1e-4621-9c2c-50225fb96baf None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:07.489256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390648b-ad1e-4621-9c2c-50225fb96baf None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:07.489256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390648b-ad1e-4621-9c2c-50225fb96baf None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:07.491147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 582/1162] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:43:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:43:07.496517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073546d-62da-4677-8692-66e381fe8004 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:07.496961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073546d-62da-4677-8692-66e381fe8004 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:07.497131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073546d-62da-4677-8692-66e381fe8004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:07.514328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-5073546d-62da-4677-8692-66e381fe8004 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:43:07.515574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5073546d-62da-4677-8692-66e381fe8004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:43:07.580819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 581/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:43:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:43:07.603024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390648b-ad1e-4621-9c2c-50225fb96baf None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:07.635142 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:07.704539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390648b-ad1e-4621-9c2c-50225fb96baf None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:07.739186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0dc28-c40a-41e1-8191-fc90afa3ae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:07.739869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0dc28-c40a-41e1-8191-fc90afa3ae36 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:07.740152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0dc28-c40a-41e1-8191-fc90afa3ae36 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:07.740431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0dc28-c40a-41e1-8191-fc90afa3ae36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:07.807949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0dc28-c40a-41e1-8191-fc90afa3ae36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:07.809425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0dc28-c40a-41e1-8191-fc90afa3ae36 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:07.838482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:07.915996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 583/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:43:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:43:07.921923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073546d-62da-4677-8692-66e381fe8004 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:08.003038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:08.018513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073546d-62da-4677-8692-66e381fe8004 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:08.052745 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e207bd-5df6-40ca-8f3b-ce951905bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:08.053540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e207bd-5df6-40ca-8f3b-ce951905bab7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:08.053824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e207bd-5df6-40ca-8f3b-ce951905bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:08.054100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e207bd-5df6-40ca-8f3b-ce951905bab7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:08.060636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e207bd-5df6-40ca-8f3b-ce951905bab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:08.062103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e207bd-5df6-40ca-8f3b-ce951905bab7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-62e207bd-5df6-40ca-8f3b-ce951905bab7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:43:08.076624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:43:08.077877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:43:08.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 582/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:43:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:43:08.084494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0dc28-c40a-41e1-8191-fc90afa3ae36 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:08.184253 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0dc28-c40a-41e1-8191-fc90afa3ae36 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:08.213025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:08.217842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c15f08-2e08-46f2-8f6f-5a12a8afa121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:08.221285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15f08-2e08-46f2-8f6f-5a12a8afa121 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:08.221285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15f08-2e08-46f2-8f6f-5a12a8afa121 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:08.221285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15f08-2e08-46f2-8f6f-5a12a8afa121 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:08.221285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c15f08-2e08-46f2-8f6f-5a12a8afa121 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:43:08.221285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c15f08-2e08-46f2-8f6f-5a12a8afa121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:08.221721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c15f08-2e08-46f2-8f6f-5a12a8afa121 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:08.230003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 584/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Jul 24 19:43:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:10.334173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779266-7acf-43cb-a6d5-faeefabb5478 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:10.334719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779266-7acf-43cb-a6d5-faeefabb5478 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:10.334719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779266-7acf-43cb-a6d5-faeefabb5478 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:10.336108 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 583/1167] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:43:10.341363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621286-94e7-4b43-b1b2-98bb2f9d4724 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:10.341747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621286-94e7-4b43-b1b2-98bb2f9d4724 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:10.341933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621286-94e7-4b43-b1b2-98bb2f9d4724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:10.359049 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-34621286-94e7-4b43-b1b2-98bb2f9d4724 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:43:10.360266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-34621286-94e7-4b43-b1b2-98bb2f9d4724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:43:10.428212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 585/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:43:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:43:10.450736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29779266-7acf-43cb-a6d5-faeefabb5478 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:10.549824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29779266-7acf-43cb-a6d5-faeefabb5478 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:10.581588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb30fa2-4af7-4c30-b85c-55bb924429b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:10.582229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb30fa2-4af7-4c30-b85c-55bb924429b4 None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:10.582512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb30fa2-4af7-4c30-b85c-55bb924429b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:10.582784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb30fa2-4af7-4c30-b85c-55bb924429b4 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:10.584024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb30fa2-4af7-4c30-b85c-55bb924429b4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:10.585459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb30fa2-4af7-4c30-b85c-55bb924429b4 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:10.709254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 584/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Sun Jul 24 19:43:10 2022] POST /identity/v3/domains => generated 267 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:43:12.770071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbc15-3187-46a8-ada1-ceeee25ea26c None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:12.770617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbc15-3187-46a8-ada1-ceeee25ea26c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:12.770617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbc15-3187-46a8-ada1-ceeee25ea26c None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:12.772523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 586/1170] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:43:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:43:12.777890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeca908-aabc-445a-bf98-13509cb1a8c2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:12.778307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeca908-aabc-445a-bf98-13509cb1a8c2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:12.778451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeca908-aabc-445a-bf98-13509cb1a8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:12.784261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:12.795686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-eaeca908-aabc-445a-bf98-13509cb1a8c2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:43:12.798653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-eaeca908-aabc-445a-bf98-13509cb1a8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:43:12.861597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 585/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:43:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:43:12.884622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cbc15-3187-46a8-ada1-ceeee25ea26c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:12.983946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cbc15-3187-46a8-ada1-ceeee25ea26c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:13.008863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:13.017024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f2c6e-7130-4917-80fb-9884f276bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:13.021770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f2c6e-7130-4917-80fb-9884f276bcde None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:13.021770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f2c6e-7130-4917-80fb-9884f276bcde None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:13.021770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f2c6e-7130-4917-80fb-9884f276bcde None admin] PATH_INFO: `/v3/domains/8646349833c0452295671f40fa6eba41` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:13.028777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f2c6e-7130-4917-80fb-9884f276bcde None admin] RBAC: Authorizing `identity:get_domain(domain_id=8646349833c0452295671f40fa6eba41)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:13.030236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f2c6e-7130-4917-80fb-9884f276bcde None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:13.031938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 587/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Sun Jul 24 19:43:12 2022] GET /identity/v3/domains/8646349833c0452295671f40fa6eba41 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:13.038386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeca908-aabc-445a-bf98-13509cb1a8c2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:13.135288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeca908-aabc-445a-bf98-13509cb1a8c2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:13.168649 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d1132-f0bb-48e8-b950-0eb7e72ab87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:13.169331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d1132-f0bb-48e8-b950-0eb7e72ab87d None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:13.169624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d1132-f0bb-48e8-b950-0eb7e72ab87d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:13.169904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d1132-f0bb-48e8-b950-0eb7e72ab87d None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:13.170613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d1132-f0bb-48e8-b950-0eb7e72ab87d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:13.172058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d1132-f0bb-48e8-b950-0eb7e72ab87d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:13.175264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:13.314845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 586/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:43:13 2022] POST /identity/v3/projects => generated 361 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:43:15.396154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dce88-09e2-479e-b228-87416381c9c1 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:15.396756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dce88-09e2-479e-b228-87416381c9c1 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:15.396756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dce88-09e2-479e-b228-87416381c9c1 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:15.398116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 588/1174] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:43:15.403476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65efa01-a1ed-4d39-92f5-6408583ccb94 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:15.403929 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65efa01-a1ed-4d39-92f5-6408583ccb94 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:15.404040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65efa01-a1ed-4d39-92f5-6408583ccb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:15.421329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d65efa01-a1ed-4d39-92f5-6408583ccb94 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:43:15.422524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d65efa01-a1ed-4d39-92f5-6408583ccb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:43:15.488835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 587/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:43:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:43:15.514030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dce88-09e2-479e-b228-87416381c9c1 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:15.613387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dce88-09e2-479e-b228-87416381c9c1 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:15.648343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aa13f-3fcd-4817-bd94-8e63e04dccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:15.649156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aa13f-3fcd-4817-bd94-8e63e04dccd2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:15.649437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aa13f-3fcd-4817-bd94-8e63e04dccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:15.649707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aa13f-3fcd-4817-bd94-8e63e04dccd2 None admin] PATH_INFO: `/v3/domains/8646349833c0452295671f40fa6eba41` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:15.650862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01aa13f-3fcd-4817-bd94-8e63e04dccd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8646349833c0452295671f40fa6eba41)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:15.652274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01aa13f-3fcd-4817-bd94-8e63e04dccd2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:15.653912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 589/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Sun Jul 24 19:43:15 2022] GET /identity/v3/domains/8646349833c0452295671f40fa6eba41 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:15.659868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65efa01-a1ed-4d39-92f5-6408583ccb94 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:15.760677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65efa01-a1ed-4d39-92f5-6408583ccb94 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:15.776860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:15.794781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b691dd5-9116-464f-9c40-ab6e83ce9984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:15.795450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b691dd5-9116-464f-9c40-ab6e83ce9984 None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:15.795734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b691dd5-9116-464f-9c40-ab6e83ce9984 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:15.796011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b691dd5-9116-464f-9c40-ab6e83ce9984 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:15.796746 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b691dd5-9116-464f-9c40-ab6e83ce9984 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:15.798172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b691dd5-9116-464f-9c40-ab6e83ce9984 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:15.910153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 588/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:43:15 2022] POST /identity/v3/users => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:43:18.019411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0189-4df1-48c7-94f0-6af3922d88ec None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:18.019938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0189-4df1-48c7-94f0-6af3922d88ec None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:18.019938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0189-4df1-48c7-94f0-6af3922d88ec None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:18.021417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 590/1178] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:43:18.026703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d273e07-65e8-455f-85db-159cd2d72ad2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:18.027155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d273e07-65e8-455f-85db-159cd2d72ad2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:18.027250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d273e07-65e8-455f-85db-159cd2d72ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:18.044460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-0d273e07-65e8-455f-85db-159cd2d72ad2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:43:18.045699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0d273e07-65e8-455f-85db-159cd2d72ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:43:18.109367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 589/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:43:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:43:18.134360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd0189-4df1-48c7-94f0-6af3922d88ec None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:18.196251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:18.240482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd0189-4df1-48c7-94f0-6af3922d88ec None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:18.274325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8b43-aefd-4bc4-9e5c-53e9d92e7796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:18.275048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8b43-aefd-4bc4-9e5c-53e9d92e7796 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:18.275332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8b43-aefd-4bc4-9e5c-53e9d92e7796 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:18.275608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8b43-aefd-4bc4-9e5c-53e9d92e7796 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:18.285486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd8b43-aefd-4bc4-9e5c-53e9d92e7796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-33dd8b43-aefd-4bc4-9e5c-53e9d92e7796 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:43:18.293918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:43:18.295050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:43:18.296200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 591/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:43:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:43:18.301275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d273e07-65e8-455f-85db-159cd2d72ad2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:18.396478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d273e07-65e8-455f-85db-159cd2d72ad2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:18.397996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:18.430827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8feb0-948b-49ae-a421-a68828abd95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:18.431503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8feb0-948b-49ae-a421-a68828abd95e None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:18.431792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8feb0-948b-49ae-a421-a68828abd95e None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:18.432075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8feb0-948b-49ae-a421-a68828abd95e None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:18.432858 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8feb0-948b-49ae-a421-a68828abd95e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:43:18.433171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8feb0-948b-49ae-a421-a68828abd95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:18.434541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8feb0-948b-49ae-a421-a68828abd95e None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:18.442534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 590/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:43:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:18.448468 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8b43-aefd-4bc4-9e5c-53e9d92e7796 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:18.548968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8b43-aefd-4bc4-9e5c-53e9d92e7796 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:18.562591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:18.581872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a616e-e346-49eb-9ac6-58283d15d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:18.582742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a616e-e346-49eb-9ac6-58283d15d15e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:18.583051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a616e-e346-49eb-9ac6-58283d15d15e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:18.583330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a616e-e346-49eb-9ac6-58283d15d15e None admin] PATH_INFO: `/v3/users/93d612fb71e24984850e056fffa1c2ac` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:18.596407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a616e-e346-49eb-9ac6-58283d15d15e None admin] RBAC: Authorizing `identity:get_user(user_id=93d612fb71e24984850e056fffa1c2ac)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:18.597898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a616e-e346-49eb-9ac6-58283d15d15e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:18.600195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 592/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:43:18 2022] GET /identity/v3/users/93d612fb71e24984850e056fffa1c2ac => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:18.606235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8feb0-948b-49ae-a421-a68828abd95e None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:18.703426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8feb0-948b-49ae-a421-a68828abd95e None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:18.738585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7867e-d735-4e8b-a0b2-e571c3df9075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:18.739358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7867e-d735-4e8b-a0b2-e571c3df9075 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:18.739649 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7867e-d735-4e8b-a0b2-e571c3df9075 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:18.739925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7867e-d735-4e8b-a0b2-e571c3df9075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:18.768986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:18.805256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7867e-d735-4e8b-a0b2-e571c3df9075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:18.806641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7867e-d735-4e8b-a0b2-e571c3df9075 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:18.900206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 591/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:43:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:43:18.906061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a616e-e346-49eb-9ac6-58283d15d15e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:19.002208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a616e-e346-49eb-9ac6-58283d15d15e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:19.035753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26167407-beec-4cbd-849b-a5bf7ad7aecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:19.036634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26167407-beec-4cbd-849b-a5bf7ad7aecd None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:19.036944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26167407-beec-4cbd-849b-a5bf7ad7aecd None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:19.037217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26167407-beec-4cbd-849b-a5bf7ad7aecd None admin] PATH_INFO: `/v3/projects/6de76a85f20f437ebd885cab472ddb52` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:19.038487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26167407-beec-4cbd-849b-a5bf7ad7aecd None admin] RBAC: Authorizing `identity:get_project(project_id=6de76a85f20f437ebd885cab472ddb52)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:19.039901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26167407-beec-4cbd-849b-a5bf7ad7aecd None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:19.041865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 593/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:43:18 2022] GET /identity/v3/projects/6de76a85f20f437ebd885cab472ddb52 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:19.048007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7867e-d735-4e8b-a0b2-e571c3df9075 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:19.107456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:19.143510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7867e-d735-4e8b-a0b2-e571c3df9075 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:19.177522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0278a24-edac-4165-8144-2e664e4863b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:19.178158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0278a24-edac-4165-8144-2e664e4863b0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:19.178445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0278a24-edac-4165-8144-2e664e4863b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:19.178719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0278a24-edac-4165-8144-2e664e4863b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:19.179539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0278a24-edac-4165-8144-2e664e4863b0 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=6de76a85f20f437ebd885cab472ddb52, user.id=93d612fb71e24984850e056fffa1c2ac) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:43:19.179849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0278a24-edac-4165-8144-2e664e4863b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:19.181310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0278a24-edac-4165-8144-2e664e4863b0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:19.186684 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 592/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:43:19 2022] GET /identity/v3/role_assignments?user.id=93d612fb71e24984850e056fffa1c2ac&scope.project.id=6de76a85f20f437ebd885cab472ddb52&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:21.194606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294bf3-449e-40eb-801d-c1e8dcb9386e None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:21.195165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294bf3-449e-40eb-801d-c1e8dcb9386e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:21.195165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294bf3-449e-40eb-801d-c1e8dcb9386e None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:21.196515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 594/1186] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:43:21.201989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a4e8-b7e3-4a23-805e-bc17cd83defd None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:21.202388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a4e8-b7e3-4a23-805e-bc17cd83defd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:21.202562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a4e8-b7e3-4a23-805e-bc17cd83defd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:21.219119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-4cb1a4e8-b7e3-4a23-805e-bc17cd83defd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:43:21.220292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4cb1a4e8-b7e3-4a23-805e-bc17cd83defd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:43:21.285629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 593/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:43:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:43:21.309408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294bf3-449e-40eb-801d-c1e8dcb9386e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:21.352063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:21.409222 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294bf3-449e-40eb-801d-c1e8dcb9386e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:21.443445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464704f5-a8af-4692-af05-f39d1e120832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:21.444334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464704f5-a8af-4692-af05-f39d1e120832 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:21.444685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464704f5-a8af-4692-af05-f39d1e120832 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:21.444915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464704f5-a8af-4692-af05-f39d1e120832 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:21.454631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464704f5-a8af-4692-af05-f39d1e120832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-464704f5-a8af-4692-af05-f39d1e120832 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:43:21.462627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:43:21.463760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:43:21.465000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 595/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:43:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:43:21.469912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1a4e8-b7e3-4a23-805e-bc17cd83defd None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:21.534462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:21.575487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1a4e8-b7e3-4a23-805e-bc17cd83defd None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:21.608806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111ff63-0b14-4876-815d-3ba96b9c0a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:21.609503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111ff63-0b14-4876-815d-3ba96b9c0a33 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:21.609767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111ff63-0b14-4876-815d-3ba96b9c0a33 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:21.610046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111ff63-0b14-4876-815d-3ba96b9c0a33 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:21.610795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111ff63-0b14-4876-815d-3ba96b9c0a33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:43:21.616142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111ff63-0b14-4876-815d-3ba96b9c0a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:21.617630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111ff63-0b14-4876-815d-3ba96b9c0a33 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:21.625818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 594/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:43:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:21.631895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464704f5-a8af-4692-af05-f39d1e120832 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:21.728028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:21.732416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464704f5-a8af-4692-af05-f39d1e120832 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:21.765415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd5cf5-7ba7-48f1-97c8-2f32715547e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:21.766270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd5cf5-7ba7-48f1-97c8-2f32715547e1 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:21.766550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd5cf5-7ba7-48f1-97c8-2f32715547e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:21.766829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd5cf5-7ba7-48f1-97c8-2f32715547e1 None admin] PATH_INFO: `/v3/users/93d612fb71e24984850e056fffa1c2ac` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:21.767992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd5cf5-7ba7-48f1-97c8-2f32715547e1 None admin] RBAC: Authorizing `identity:get_user(user_id=93d612fb71e24984850e056fffa1c2ac)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:21.769467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd5cf5-7ba7-48f1-97c8-2f32715547e1 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:21.771650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 596/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:43:21 2022] GET /identity/v3/users/93d612fb71e24984850e056fffa1c2ac => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:21.777576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111ff63-0b14-4876-815d-3ba96b9c0a33 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:21.875567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111ff63-0b14-4876-815d-3ba96b9c0a33 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:21.894915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:21.908103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4783398-eab1-4005-8374-e50cf0fddeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:21.908987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4783398-eab1-4005-8374-e50cf0fddeba None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:21.909268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4783398-eab1-4005-8374-e50cf0fddeba None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:21.909546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4783398-eab1-4005-8374-e50cf0fddeba None admin] PATH_INFO: `/v3/projects/6de76a85f20f437ebd885cab472ddb52` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:21.910711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4783398-eab1-4005-8374-e50cf0fddeba None admin] RBAC: Authorizing `identity:get_project(project_id=6de76a85f20f437ebd885cab472ddb52)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:21.912156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4783398-eab1-4005-8374-e50cf0fddeba None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:21.913874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 595/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:43:21 2022] GET /identity/v3/projects/6de76a85f20f437ebd885cab472ddb52 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:21.920236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd5cf5-7ba7-48f1-97c8-2f32715547e1 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:22.027854 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd5cf5-7ba7-48f1-97c8-2f32715547e1 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:22.061819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b50315-1d63-460a-916a-25b05dc40b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:22.062696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b50315-1d63-460a-916a-25b05dc40b54 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:22.062984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b50315-1d63-460a-916a-25b05dc40b54 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:22.063260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b50315-1d63-460a-916a-25b05dc40b54 None admin] PATH_INFO: `/v3/projects/6de76a85f20f437ebd885cab472ddb52/users/93d612fb71e24984850e056fffa1c2ac/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:22.065214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b50315-1d63-460a-916a-25b05dc40b54 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de76a85f20f437ebd885cab472ddb52, user_id=93d612fb71e24984850e056fffa1c2ac, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:22.066639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b50315-1d63-460a-916a-25b05dc40b54 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:22.209691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 597/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:43:21 2022] PUT /identity/v3/projects/6de76a85f20f437ebd885cab472ddb52/users/93d612fb71e24984850e056fffa1c2ac/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:43:24.208669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682382a2-4ffc-411b-a5ad-42420e7423c0 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:24.209245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682382a2-4ffc-411b-a5ad-42420e7423c0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:24.209245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682382a2-4ffc-411b-a5ad-42420e7423c0 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:24.210588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 596/1193] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:43:24.215784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb13ce1-6c95-41f2-a2d5-6bc761cef178 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:24.216196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb13ce1-6c95-41f2-a2d5-6bc761cef178 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:24.216372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb13ce1-6c95-41f2-a2d5-6bc761cef178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:24.233521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-1eb13ce1-6c95-41f2-a2d5-6bc761cef178 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:43:24.234689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1eb13ce1-6c95-41f2-a2d5-6bc761cef178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:43:24.257735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:24.300527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 598/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:43:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:43:24.324417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682382a2-4ffc-411b-a5ad-42420e7423c0 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:24.423337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682382a2-4ffc-411b-a5ad-42420e7423c0 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:24.456009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06521d6-3af8-4380-8e3a-06f03ce2f43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:24.456828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06521d6-3af8-4380-8e3a-06f03ce2f43f None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:24.457113 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06521d6-3af8-4380-8e3a-06f03ce2f43f None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:24.457384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06521d6-3af8-4380-8e3a-06f03ce2f43f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:24.467702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06521d6-3af8-4380-8e3a-06f03ce2f43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-d06521d6-3af8-4380-8e3a-06f03ce2f43f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:43:24.475267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:43:24.476331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:43:24.477544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 597/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:43:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:43:24.482531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb13ce1-6c95-41f2-a2d5-6bc761cef178 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:24.578374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb13ce1-6c95-41f2-a2d5-6bc761cef178 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:24.611479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b00047-1399-4443-a5e3-fc4f748d4315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:24.612155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b00047-1399-4443-a5e3-fc4f748d4315 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:24.612445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b00047-1399-4443-a5e3-fc4f748d4315 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:24.612753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b00047-1399-4443-a5e3-fc4f748d4315 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:24.613511 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b00047-1399-4443-a5e3-fc4f748d4315 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:43:24.614328 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b00047-1399-4443-a5e3-fc4f748d4315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:24.615692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b00047-1399-4443-a5e3-fc4f748d4315 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:24.623674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 599/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:43:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:24.630202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06521d6-3af8-4380-8e3a-06f03ce2f43f None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:24.678514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:24.727583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06521d6-3af8-4380-8e3a-06f03ce2f43f None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:24.764921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7546f5-8d70-423c-abb8-bc353b8d4023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:24.765784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7546f5-8d70-423c-abb8-bc353b8d4023 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:24.766067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7546f5-8d70-423c-abb8-bc353b8d4023 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:24.766341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7546f5-8d70-423c-abb8-bc353b8d4023 None admin] PATH_INFO: `/v3/users/93d612fb71e24984850e056fffa1c2ac` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:24.767511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7546f5-8d70-423c-abb8-bc353b8d4023 None admin] RBAC: Authorizing `identity:get_user(user_id=93d612fb71e24984850e056fffa1c2ac)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:24.768955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7546f5-8d70-423c-abb8-bc353b8d4023 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:24.770605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 598/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Jul 24 19:43:24 2022] GET /identity/v3/users/93d612fb71e24984850e056fffa1c2ac => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:24.776532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b00047-1399-4443-a5e3-fc4f748d4315 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:24.798545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:24.875454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b00047-1399-4443-a5e3-fc4f748d4315 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:24.908756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab2aa1-4d47-4e59-a0a5-5c450c9fe89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:24.909399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab2aa1-4d47-4e59-a0a5-5c450c9fe89a None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:24.909681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab2aa1-4d47-4e59-a0a5-5c450c9fe89a None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:24.909958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab2aa1-4d47-4e59-a0a5-5c450c9fe89a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:24.976739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab2aa1-4d47-4e59-a0a5-5c450c9fe89a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:24.978306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab2aa1-4d47-4e59-a0a5-5c450c9fe89a None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:24.997716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:25.074117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 600/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:43:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:43:25.080177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7546f5-8d70-423c-abb8-bc353b8d4023 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:25.178500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7546f5-8d70-423c-abb8-bc353b8d4023 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:25.195378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:25.218400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fdcd9-b728-44c5-bbb7-1d9490973093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:25.219266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fdcd9-b728-44c5-bbb7-1d9490973093 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:25.219555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fdcd9-b728-44c5-bbb7-1d9490973093 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:25.219835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fdcd9-b728-44c5-bbb7-1d9490973093 None admin] PATH_INFO: `/v3/projects/6de76a85f20f437ebd885cab472ddb52` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:25.221080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fdcd9-b728-44c5-bbb7-1d9490973093 None admin] RBAC: Authorizing `identity:get_project(project_id=6de76a85f20f437ebd885cab472ddb52)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:25.226255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fdcd9-b728-44c5-bbb7-1d9490973093 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:25.226255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 599/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Jul 24 19:43:25 2022] GET /identity/v3/projects/6de76a85f20f437ebd885cab472ddb52 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:25.230570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab2aa1-4d47-4e59-a0a5-5c450c9fe89a None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:43:25.328462 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab2aa1-4d47-4e59-a0a5-5c450c9fe89a None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:43:25.348626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:25.361905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b40ba-8021-41fc-8af5-e39029923beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:43:25.362557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b40ba-8021-41fc-8af5-e39029923beb None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:43:25.362842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b40ba-8021-41fc-8af5-e39029923beb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:43:25.363120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b40ba-8021-41fc-8af5-e39029923beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:43:25.364443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b40ba-8021-41fc-8af5-e39029923beb None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=6de76a85f20f437ebd885cab472ddb52, user.id=93d612fb71e24984850e056fffa1c2ac) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:43:25.364823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b40ba-8021-41fc-8af5-e39029923beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:43:25.366344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b40ba-8021-41fc-8af5-e39029923beb None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:43:25.372289 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 601/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:43:25 2022] GET /identity/v3/role_assignments?user.id=93d612fb71e24984850e056fffa1c2ac&scope.project.id=6de76a85f20f437ebd885cab472ddb52&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:43:35.247010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=8 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:43:35.378376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=8 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:40.479517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a58b9c-2033-4c87-9694-4cb05670c856 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:40.480168 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a58b9c-2033-4c87-9694-4cb05670c856 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:40.480168 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a58b9c-2033-4c87-9694-4cb05670c856 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:40.481495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 600/1201] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:44:40.487681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207ca1e-5c47-45a4-aa6b-7f6a88259be7 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:40.488242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207ca1e-5c47-45a4-aa6b-7f6a88259be7 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:40.488242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207ca1e-5c47-45a4-aa6b-7f6a88259be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:40.506408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-2207ca1e-5c47-45a4-aa6b-7f6a88259be7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:44:40.513124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2207ca1e-5c47-45a4-aa6b-7f6a88259be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:44:40.585330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 602/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:44:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:44:40.610977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a58b9c-2033-4c87-9694-4cb05670c856 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:40.719496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a58b9c-2033-4c87-9694-4cb05670c856 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:40.756861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69938043-3c65-4f9f-9aea-f2c3d2a4d562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:40.762098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69938043-3c65-4f9f-9aea-f2c3d2a4d562 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:40.762098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69938043-3c65-4f9f-9aea-f2c3d2a4d562 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:40.762098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69938043-3c65-4f9f-9aea-f2c3d2a4d562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:40.769610 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69938043-3c65-4f9f-9aea-f2c3d2a4d562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:40.771067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69938043-3c65-4f9f-9aea-f2c3d2a4d562 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-69938043-3c65-4f9f-9aea-f2c3d2a4d562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:44:40.789539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:40.791130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:40.792706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:44:40.792706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:44:40.792706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:40.792706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:44:40.792706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 601/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:44:40.797425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207ca1e-5c47-45a4-aa6b-7f6a88259be7 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:40.914654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207ca1e-5c47-45a4-aa6b-7f6a88259be7 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:40.946133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:40.952314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d50e62-d0c2-4c44-b11c-8a45b003d47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:40.953016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d50e62-d0c2-4c44-b11c-8a45b003d47f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:40.953847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d50e62-d0c2-4c44-b11c-8a45b003d47f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:40.954132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d50e62-d0c2-4c44-b11c-8a45b003d47f None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:40.954866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d50e62-d0c2-4c44-b11c-8a45b003d47f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:40.955174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d50e62-d0c2-4c44-b11c-8a45b003d47f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:40.956559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d50e62-d0c2-4c44-b11c-8a45b003d47f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:40.968735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 603/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:44:40.975095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69938043-3c65-4f9f-9aea-f2c3d2a4d562 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:41.037555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:41.096014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69938043-3c65-4f9f-9aea-f2c3d2a4d562 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:41.134160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e3a4f-a55e-4405-823f-9b3d8f507bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:41.134840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e3a4f-a55e-4405-823f-9b3d8f507bb6 None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:41.135122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e3a4f-a55e-4405-823f-9b3d8f507bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:41.135399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e3a4f-a55e-4405-823f-9b3d8f507bb6 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:41.136131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e3a4f-a55e-4405-823f-9b3d8f507bb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:41.137615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e3a4f-a55e-4405-823f-9b3d8f507bb6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:41.251493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 602/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Jul 24 19:44:40 2022] POST /identity/v3/users => generated 251 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 19:44:43.437221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cd05d-e99a-4820-bdce-45e798f1f2d0 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:43.438059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cd05d-e99a-4820-bdce-45e798f1f2d0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:43.438059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cd05d-e99a-4820-bdce-45e798f1f2d0 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:43.439345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 604/1206] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:44:43.445272 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed0712-90da-4682-912a-d715690ccc7f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:43.446123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed0712-90da-4682-912a-d715690ccc7f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:43.446348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed0712-90da-4682-912a-d715690ccc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:43.464573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c0ed0712-90da-4682-912a-d715690ccc7f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:44:43.468485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c0ed0712-90da-4682-912a-d715690ccc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:44:43.540318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 603/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:44:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:44:43.564776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cd05d-e99a-4820-bdce-45e798f1f2d0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:43.724361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cd05d-e99a-4820-bdce-45e798f1f2d0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:43.783531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7971847-6204-4c55-8d6c-00d7dce910aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:43.784942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7971847-6204-4c55-8d6c-00d7dce910aa None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:43.785432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7971847-6204-4c55-8d6c-00d7dce910aa None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:43.785898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7971847-6204-4c55-8d6c-00d7dce910aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:43.805345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7971847-6204-4c55-8d6c-00d7dce910aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-f7971847-6204-4c55-8d6c-00d7dce910aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:44:43.821256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:43.823040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:43.824730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 605/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:44:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:44:43.830252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed0712-90da-4682-912a-d715690ccc7f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:43.874578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:43.937084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed0712-90da-4682-912a-d715690ccc7f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:43.983024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe953fe-6a0a-45f8-a628-84ac3ba828f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:43.983712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe953fe-6a0a-45f8-a628-84ac3ba828f2 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:43.983999 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe953fe-6a0a-45f8-a628-84ac3ba828f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:43.984279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe953fe-6a0a-45f8-a628-84ac3ba828f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:43.985354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe953fe-6a0a-45f8-a628-84ac3ba828f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:43.985669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe953fe-6a0a-45f8-a628-84ac3ba828f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:43.987053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe953fe-6a0a-45f8-a628-84ac3ba828f2 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:43.996054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 604/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:44:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:44.002920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7971847-6204-4c55-8d6c-00d7dce910aa None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:44.059111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:44.120564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7971847-6204-4c55-8d6c-00d7dce910aa None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:44.155542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1312f8e-dfba-4e68-aaf4-0155f8e86ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:44.156341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1312f8e-dfba-4e68-aaf4-0155f8e86ed7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:44.156621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1312f8e-dfba-4e68-aaf4-0155f8e86ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:44.156924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1312f8e-dfba-4e68-aaf4-0155f8e86ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:44.168451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1312f8e-dfba-4e68-aaf4-0155f8e86ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:44.170102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1312f8e-dfba-4e68-aaf4-0155f8e86ed7 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-e1312f8e-dfba-4e68-aaf4-0155f8e86ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:44:44.190230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:44.196859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:44.197800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:44:44.197800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:44:44.197800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:44.197800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:44.197800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 606/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:44:44.198316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe953fe-6a0a-45f8-a628-84ac3ba828f2 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:44.324358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:44.329682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe953fe-6a0a-45f8-a628-84ac3ba828f2 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:44.380465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6f198-0ebc-4ea4-96e4-b981848cc681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:44.381341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6f198-0ebc-4ea4-96e4-b981848cc681 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:44.381715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6f198-0ebc-4ea4-96e4-b981848cc681 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:44.382149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6f198-0ebc-4ea4-96e4-b981848cc681 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:44.382986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6f198-0ebc-4ea4-96e4-b981848cc681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:44.383390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6f198-0ebc-4ea4-96e4-b981848cc681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:44.384989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6f198-0ebc-4ea4-96e4-b981848cc681 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:44.398063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 605/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:44.404553 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1312f8e-dfba-4e68-aaf4-0155f8e86ed7 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:44.529174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1312f8e-dfba-4e68-aaf4-0155f8e86ed7 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:44.530952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:44.566879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578be49-1e78-4af1-a0ed-837dfbd8edeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:44.567699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578be49-1e78-4af1-a0ed-837dfbd8edeb None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:44.567990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578be49-1e78-4af1-a0ed-837dfbd8edeb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:44.568270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578be49-1e78-4af1-a0ed-837dfbd8edeb None admin] PATH_INFO: `/v3/users/placement` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:44.576356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578be49-1e78-4af1-a0ed-837dfbd8edeb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:44.577877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578be49-1e78-4af1-a0ed-837dfbd8edeb None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-0578be49-1e78-4af1-a0ed-837dfbd8edeb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:44:44.607696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:44.610155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:44.619416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 607/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:44:44.620580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6f198-0ebc-4ea4-96e4-b981848cc681 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:44.748499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6f198-0ebc-4ea4-96e4-b981848cc681 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:44.792333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0d53e-b27c-4767-afbf-afe76d0e5641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:44.793604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0d53e-b27c-4767-afbf-afe76d0e5641 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:44.794117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0d53e-b27c-4767-afbf-afe76d0e5641 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:44.794619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0d53e-b27c-4767-afbf-afe76d0e5641 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:44.796233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0d53e-b27c-4767-afbf-afe76d0e5641 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:44.797459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0d53e-b27c-4767-afbf-afe76d0e5641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:44.800266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0d53e-b27c-4767-afbf-afe76d0e5641 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:44.832872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 606/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Jul 24 19:44:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:44:44.840280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578be49-1e78-4af1-a0ed-837dfbd8edeb None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:45.021232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578be49-1e78-4af1-a0ed-837dfbd8edeb None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:45.087374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d62222-5b22-4769-811d-f4bf1274cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:45.088106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d62222-5b22-4769-811d-f4bf1274cdfe None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:45.088395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d62222-5b22-4769-811d-f4bf1274cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:45.088790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d62222-5b22-4769-811d-f4bf1274cdfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:45.127168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:45.169229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d62222-5b22-4769-811d-f4bf1274cdfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:45.172102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d62222-5b22-4769-811d-f4bf1274cdfe None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:45.293344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 608/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:44:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:44:45.299802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0d53e-b27c-4767-afbf-afe76d0e5641 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:45.321773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:45.430285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0d53e-b27c-4767-afbf-afe76d0e5641 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:45.469206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ffae6-1ad4-4cb8-a2c4-86b8c2a14576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:45.470024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ffae6-1ad4-4cb8-a2c4-86b8c2a14576 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:45.470310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ffae6-1ad4-4cb8-a2c4-86b8c2a14576 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:45.470585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ffae6-1ad4-4cb8-a2c4-86b8c2a14576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:45.482327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ffae6-1ad4-4cb8-a2c4-86b8c2a14576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:45.484026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ffae6-1ad4-4cb8-a2c4-86b8c2a14576 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-ed0ffae6-1ad4-4cb8-a2c4-86b8c2a14576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:44:45.504523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:45.505600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:45.507264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:44:45.507264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:44:45.507264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:45.507264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:44:45.507264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 607/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:45.512610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d62222-5b22-4769-811d-f4bf1274cdfe None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:45.581852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:45.623971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d62222-5b22-4769-811d-f4bf1274cdfe None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:45.666450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86e210-09a1-420c-a275-cceb7807f5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:45.667239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86e210-09a1-420c-a275-cceb7807f5e3 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:45.667629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86e210-09a1-420c-a275-cceb7807f5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:45.667950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86e210-09a1-420c-a275-cceb7807f5e3 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:45.668777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86e210-09a1-420c-a275-cceb7807f5e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:45.669133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86e210-09a1-420c-a275-cceb7807f5e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:45.670665 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86e210-09a1-420c-a275-cceb7807f5e3 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:45.685056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 609/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:45.691474 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ffae6-1ad4-4cb8-a2c4-86b8c2a14576 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:45.791964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:45.824923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ffae6-1ad4-4cb8-a2c4-86b8c2a14576 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:45.862032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0949db8-5313-4630-8311-eb256566a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:45.862837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0949db8-5313-4630-8311-eb256566a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:45.863122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0949db8-5313-4630-8311-eb256566a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:45.863403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0949db8-5313-4630-8311-eb256566a2e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:45.871112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0949db8-5313-4630-8311-eb256566a2e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:45.872638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0949db8-5313-4630-8311-eb256566a2e6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-b0949db8-5313-4630-8311-eb256566a2e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:44:45.888010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:45.889645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:45.891602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:44:45.891602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:44:45.891602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:44:45.891602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:44:45.891602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:44:45.891602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:44:45.891602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:44:45.891602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:44:45.891602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 608/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:44:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:45.897294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86e210-09a1-420c-a275-cceb7807f5e3 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:46.015999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86e210-09a1-420c-a275-cceb7807f5e3 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:46.041896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:46.054236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc684c-9c0d-4dec-9fc2-80fff818fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:46.054984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc684c-9c0d-4dec-9fc2-80fff818fe9e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:46.055315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc684c-9c0d-4dec-9fc2-80fff818fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:46.055649 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc684c-9c0d-4dec-9fc2-80fff818fe9e None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:46.056499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc684c-9c0d-4dec-9fc2-80fff818fe9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:46.056876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc684c-9c0d-4dec-9fc2-80fff818fe9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:46.058481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc684c-9c0d-4dec-9fc2-80fff818fe9e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:46.072078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 610/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:44:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:46.078567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0949db8-5313-4630-8311-eb256566a2e6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:46.191785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0949db8-5313-4630-8311-eb256566a2e6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:46.224340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:46.230388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5c1f9-b17d-4924-9326-995ce9bde66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:46.231049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c1f9-b17d-4924-9326-995ce9bde66b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:46.231333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c1f9-b17d-4924-9326-995ce9bde66b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:46.231609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c1f9-b17d-4924-9326-995ce9bde66b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:46.232441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5c1f9-b17d-4924-9326-995ce9bde66b None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=1c1b23b83544423abc6919d805916b8d) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:46.232782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5c1f9-b17d-4924-9326-995ce9bde66b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:46.234827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5c1f9-b17d-4924-9326-995ce9bde66b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:46.240546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 609/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:44:46 2022] GET /identity/v3/role_assignments?user.id=1c1b23b83544423abc6919d805916b8d&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:48.369749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53175c5-7b35-4c51-8b64-9ec8771178a7 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:48.370445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53175c5-7b35-4c51-8b64-9ec8771178a7 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:48.370445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53175c5-7b35-4c51-8b64-9ec8771178a7 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:48.371671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 611/1220] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:44:48.378502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e800f4-c57f-49a0-a7d2-38abd7b826b6 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:48.381390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e800f4-c57f-49a0-a7d2-38abd7b826b6 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:48.381390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e800f4-c57f-49a0-a7d2-38abd7b826b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:48.399713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c8e800f4-c57f-49a0-a7d2-38abd7b826b6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:44:48.400962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c8e800f4-c57f-49a0-a7d2-38abd7b826b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:44:48.480484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 610/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:44:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:44:48.505733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53175c5-7b35-4c51-8b64-9ec8771178a7 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:48.624626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53175c5-7b35-4c51-8b64-9ec8771178a7 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:48.670154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032851d-7c69-4e7c-8244-1fae87ce34d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:48.671056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032851d-7c69-4e7c-8244-1fae87ce34d5 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:48.671440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032851d-7c69-4e7c-8244-1fae87ce34d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:48.672299 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032851d-7c69-4e7c-8244-1fae87ce34d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:48.685201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032851d-7c69-4e7c-8244-1fae87ce34d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-b032851d-7c69-4e7c-8244-1fae87ce34d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:44:48.693886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:48.698351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:48.700225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 612/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:44:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:48.702590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e800f4-c57f-49a0-a7d2-38abd7b826b6 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:48.842315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e800f4-c57f-49a0-a7d2-38abd7b826b6 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:48.893118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:48.904563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90624c07-8297-44e4-8b63-d93af1d0fe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:48.905286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90624c07-8297-44e4-8b63-d93af1d0fe30 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:48.905571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90624c07-8297-44e4-8b63-d93af1d0fe30 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:48.905885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90624c07-8297-44e4-8b63-d93af1d0fe30 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:48.906671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90624c07-8297-44e4-8b63-d93af1d0fe30 None admin] RBAC: Adding query filter params (name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:48.906978 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90624c07-8297-44e4-8b63-d93af1d0fe30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:48.908390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90624c07-8297-44e4-8b63-d93af1d0fe30 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:48.918247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 611/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:44:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:48.925235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032851d-7c69-4e7c-8244-1fae87ce34d5 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:48.977539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:49.044122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032851d-7c69-4e7c-8244-1fae87ce34d5 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:49.085133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e6e6e-4c59-464e-b7f8-eb67544175d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:49.086651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e6e6e-4c59-464e-b7f8-eb67544175d3 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:49.087119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e6e6e-4c59-464e-b7f8-eb67544175d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:49.087596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e6e6e-4c59-464e-b7f8-eb67544175d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:49.101735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e6e6e-4c59-464e-b7f8-eb67544175d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:49.103219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e6e6e-4c59-464e-b7f8-eb67544175d3 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-296e6e6e-4c59-464e-b7f8-eb67544175d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:44:49.122839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:49.126184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:49.128217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:44:49.128217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:44:49.128217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:49.128217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:49.128217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 613/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:49.130607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90624c07-8297-44e4-8b63-d93af1d0fe30 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:49.244820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90624c07-8297-44e4-8b63-d93af1d0fe30 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:49.297730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04dab0-5d0c-455c-a37f-15c988c3acff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:49.299022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04dab0-5d0c-455c-a37f-15c988c3acff None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:49.299502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04dab0-5d0c-455c-a37f-15c988c3acff None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:49.299951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04dab0-5d0c-455c-a37f-15c988c3acff None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:49.301391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04dab0-5d0c-455c-a37f-15c988c3acff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:49.301903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04dab0-5d0c-455c-a37f-15c988c3acff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:49.304944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04dab0-5d0c-455c-a37f-15c988c3acff None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:49.327679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 612/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:49.334353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e6e6e-4c59-464e-b7f8-eb67544175d3 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:49.448593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:49.456671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e6e6e-4c59-464e-b7f8-eb67544175d3 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:49.494676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5923ebb-76ff-440b-bcaa-73f4435f9304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:49.495631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5923ebb-76ff-440b-bcaa-73f4435f9304 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:49.495905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5923ebb-76ff-440b-bcaa-73f4435f9304 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:49.496223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5923ebb-76ff-440b-bcaa-73f4435f9304 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:49.505461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5923ebb-76ff-440b-bcaa-73f4435f9304 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:49.506935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5923ebb-76ff-440b-bcaa-73f4435f9304 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-b5923ebb-76ff-440b-bcaa-73f4435f9304 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:44:49.523636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:49.527711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:49.529489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 614/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:49.533853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04dab0-5d0c-455c-a37f-15c988c3acff None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:49.585703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:49.642580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04dab0-5d0c-455c-a37f-15c988c3acff None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:49.680313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87b47b-8f52-479a-a019-a2038dcf0dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:49.681030 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87b47b-8f52-479a-a019-a2038dcf0dfb None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:49.681317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87b47b-8f52-479a-a019-a2038dcf0dfb None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:49.681596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87b47b-8f52-479a-a019-a2038dcf0dfb None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:49.682457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87b47b-8f52-479a-a019-a2038dcf0dfb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:49.682773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87b47b-8f52-479a-a019-a2038dcf0dfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:49.684238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87b47b-8f52-479a-a019-a2038dcf0dfb None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:49.704123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 613/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Jul 24 19:44:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:49.710310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5923ebb-76ff-440b-bcaa-73f4435f9304 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:49.830547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5923ebb-76ff-440b-bcaa-73f4435f9304 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:49.866228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa17ce-e2b2-4876-aa01-0a068214bf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:49.867112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa17ce-e2b2-4876-aa01-0a068214bf9c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:49.867550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa17ce-e2b2-4876-aa01-0a068214bf9c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:49.867880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa17ce-e2b2-4876-aa01-0a068214bf9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:49.880509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa17ce-e2b2-4876-aa01-0a068214bf9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:49.882370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa17ce-e2b2-4876-aa01-0a068214bf9c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-d9fa17ce-e2b2-4876-aa01-0a068214bf9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:44:49.901698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:49.903410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:49.905435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:44:49.905435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:44:49.905435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:49.905435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:49.905435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 615/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:49.910687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87b47b-8f52-479a-a019-a2038dcf0dfb None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:50.023465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87b47b-8f52-479a-a019-a2038dcf0dfb None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:50.026208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:50.062886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e6275-0a9a-4161-ad50-a4f04e06c521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:50.063611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e6275-0a9a-4161-ad50-a4f04e06c521 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:50.064236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e6275-0a9a-4161-ad50-a4f04e06c521 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:50.064938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e6275-0a9a-4161-ad50-a4f04e06c521 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:50.066009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e6275-0a9a-4161-ad50-a4f04e06c521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:50.066817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e6275-0a9a-4161-ad50-a4f04e06c521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:50.068471 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e6275-0a9a-4161-ad50-a4f04e06c521 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:50.085504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 614/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:44:50.089983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa17ce-e2b2-4876-aa01-0a068214bf9c None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:50.100473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:50.206610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa17ce-e2b2-4876-aa01-0a068214bf9c None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:50.244196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb69ee-9fac-4c6c-b5a8-a567598071fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:50.245099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb69ee-9fac-4c6c-b5a8-a567598071fa None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:50.245427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb69ee-9fac-4c6c-b5a8-a567598071fa None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:50.245748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb69ee-9fac-4c6c-b5a8-a567598071fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:50.253862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb69ee-9fac-4c6c-b5a8-a567598071fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:50.255310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb69ee-9fac-4c6c-b5a8-a567598071fa None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-f8bb69ee-9fac-4c6c-b5a8-a567598071fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:44:50.270675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:50.272332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:50.274381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:44:50.274381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:44:50.274381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:44:50.274381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:44:50.274381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:44:50.274381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:44:50.274381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:44:50.274381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:50.274381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 616/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:44:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:44:50.279615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e6275-0a9a-4161-ad50-a4f04e06c521 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:50.404596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e6275-0a9a-4161-ad50-a4f04e06c521 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:50.453196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb405ee7-6e2e-40a2-8bcc-a1ce69700a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:50.453940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb405ee7-6e2e-40a2-8bcc-a1ce69700a59 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:50.454224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb405ee7-6e2e-40a2-8bcc-a1ce69700a59 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:50.454500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb405ee7-6e2e-40a2-8bcc-a1ce69700a59 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:50.455322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb405ee7-6e2e-40a2-8bcc-a1ce69700a59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:50.455633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb405ee7-6e2e-40a2-8bcc-a1ce69700a59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:50.457173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb405ee7-6e2e-40a2-8bcc-a1ce69700a59 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:50.473826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 615/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:44:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:44:50.483640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb69ee-9fac-4c6c-b5a8-a567598071fa None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:50.561912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:50.600872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb69ee-9fac-4c6c-b5a8-a567598071fa None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:50.641939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af37c5-f423-4c4d-8fc7-c0cf7eae46f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:50.642876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af37c5-f423-4c4d-8fc7-c0cf7eae46f3 None admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:50.643164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af37c5-f423-4c4d-8fc7-c0cf7eae46f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:50.643441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af37c5-f423-4c4d-8fc7-c0cf7eae46f3 None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/1c1b23b83544423abc6919d805916b8d/roles/65e4ab8e291442e3858317b0aa6c2d8a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:50.662132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af37c5-f423-4c4d-8fc7-c0cf7eae46f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=1c1b23b83544423abc6919d805916b8d, role_id=65e4ab8e291442e3858317b0aa6c2d8a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:50.663922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af37c5-f423-4c4d-8fc7-c0cf7eae46f3 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:50.846589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 617/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:44:50 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/1c1b23b83544423abc6919d805916b8d/roles/65e4ab8e291442e3858317b0aa6c2d8a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:44:53.028089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3356c8-5be3-4c2a-905e-88897118996f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:53.028679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3356c8-5be3-4c2a-905e-88897118996f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:53.028959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3356c8-5be3-4c2a-905e-88897118996f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:53.031503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 616/1233] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:44:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:44:53.037603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3bea9-7a09-4a51-bbb3-e5154118ef27 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:53.038981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3bea9-7a09-4a51-bbb3-e5154118ef27 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:53.039412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3bea9-7a09-4a51-bbb3-e5154118ef27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:53.067436 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c5d3bea9-7a09-4a51-bbb3-e5154118ef27 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:44:53.069541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c5d3bea9-7a09-4a51-bbb3-e5154118ef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:44:53.158016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 618/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:44:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:44:53.184022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3356c8-5be3-4c2a-905e-88897118996f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:53.213594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:53.306386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3356c8-5be3-4c2a-905e-88897118996f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:53.357553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86873537-a5a6-4a32-904b-3543583578d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:53.359188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86873537-a5a6-4a32-904b-3543583578d6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:53.359668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86873537-a5a6-4a32-904b-3543583578d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:53.360115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86873537-a5a6-4a32-904b-3543583578d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:53.376375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86873537-a5a6-4a32-904b-3543583578d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-86873537-a5a6-4a32-904b-3543583578d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:44:53.384567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:53.386395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:44:53.388569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 617/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Jul 24 19:44:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:53.393697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3bea9-7a09-4a51-bbb3-e5154118ef27 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:53.467118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:53.539826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3bea9-7a09-4a51-bbb3-e5154118ef27 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:53.581619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a41ff-4a28-4a79-b068-6b65ccfd74f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:53.582276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a41ff-4a28-4a79-b068-6b65ccfd74f2 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:53.582701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a41ff-4a28-4a79-b068-6b65ccfd74f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:53.583116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a41ff-4a28-4a79-b068-6b65ccfd74f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:53.583930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a41ff-4a28-4a79-b068-6b65ccfd74f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:53.584470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a41ff-4a28-4a79-b068-6b65ccfd74f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:53.585935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a41ff-4a28-4a79-b068-6b65ccfd74f2 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:53.597886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 619/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Jul 24 19:44:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:53.604186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86873537-a5a6-4a32-904b-3543583578d6 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:53.699749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:53.716574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86873537-a5a6-4a32-904b-3543583578d6 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:53.753292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad0046-c64c-4ce3-8932-821a1c2df9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:53.754098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0046-c64c-4ce3-8932-821a1c2df9b9 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:53.754382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0046-c64c-4ce3-8932-821a1c2df9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:53.754654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0046-c64c-4ce3-8932-821a1c2df9b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:53.766277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad0046-c64c-4ce3-8932-821a1c2df9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:53.767729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad0046-c64c-4ce3-8932-821a1c2df9b9 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-e5ad0046-c64c-4ce3-8932-821a1c2df9b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:44:53.786981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:53.788991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:53.790752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:44:53.790752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:44:53.790752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:53.790752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:44:53.790752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 618/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:53.796103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a41ff-4a28-4a79-b068-6b65ccfd74f2 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:53.915458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a41ff-4a28-4a79-b068-6b65ccfd74f2 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:53.926359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:53.953198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469990b-9a87-4a15-9d95-1b6124430aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:53.953875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469990b-9a87-4a15-9d95-1b6124430aaa None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:53.954161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469990b-9a87-4a15-9d95-1b6124430aaa None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:53.954436 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469990b-9a87-4a15-9d95-1b6124430aaa None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:53.955174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469990b-9a87-4a15-9d95-1b6124430aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:53.955555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469990b-9a87-4a15-9d95-1b6124430aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:53.956888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469990b-9a87-4a15-9d95-1b6124430aaa None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:53.969250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 620/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:44:53.978133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad0046-c64c-4ce3-8932-821a1c2df9b9 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:54.088060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad0046-c64c-4ce3-8932-821a1c2df9b9 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:54.123647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c884c-6665-4d9f-b1d1-2f63caf6be99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:54.125075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c884c-6665-4d9f-b1d1-2f63caf6be99 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:54.125373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c884c-6665-4d9f-b1d1-2f63caf6be99 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:54.125755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c884c-6665-4d9f-b1d1-2f63caf6be99 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:54.128749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:54.133775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c884c-6665-4d9f-b1d1-2f63caf6be99 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:54.135334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c884c-6665-4d9f-b1d1-2f63caf6be99 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-7d5c884c-6665-4d9f-b1d1-2f63caf6be99 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:44:54.150835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:54.152386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:44:54.153495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 619/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:54.158726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469990b-9a87-4a15-9d95-1b6124430aaa None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:54.275093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469990b-9a87-4a15-9d95-1b6124430aaa None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:54.315945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f06079-e14e-4445-b8c6-2e9e16834709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:54.317482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f06079-e14e-4445-b8c6-2e9e16834709 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:54.318025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f06079-e14e-4445-b8c6-2e9e16834709 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:54.318515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f06079-e14e-4445-b8c6-2e9e16834709 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:54.320098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f06079-e14e-4445-b8c6-2e9e16834709 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:54.320757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f06079-e14e-4445-b8c6-2e9e16834709 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:54.323568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f06079-e14e-4445-b8c6-2e9e16834709 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:54.345834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 621/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Jul 24 19:44:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:54.352111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c884c-6665-4d9f-b1d1-2f63caf6be99 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:54.471787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c884c-6665-4d9f-b1d1-2f63caf6be99 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:54.509684 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd49c50-888a-4e46-99f6-c9e16c5d39fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:54.510409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd49c50-888a-4e46-99f6-c9e16c5d39fa None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:54.510737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd49c50-888a-4e46-99f6-c9e16c5d39fa None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:54.511058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd49c50-888a-4e46-99f6-c9e16c5d39fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:54.578346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:54.585343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd49c50-888a-4e46-99f6-c9e16c5d39fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:54.586804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd49c50-888a-4e46-99f6-c9e16c5d39fa None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:54.698251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 620/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:44:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 24 19:44:54.705695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f06079-e14e-4445-b8c6-2e9e16834709 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:54.738873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:54.821087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f06079-e14e-4445-b8c6-2e9e16834709 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:54.872747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cf2c7-2f9e-4c44-856a-7a43b9c756c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:54.874335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cf2c7-2f9e-4c44-856a-7a43b9c756c6 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:54.874846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cf2c7-2f9e-4c44-856a-7a43b9c756c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:54.875347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cf2c7-2f9e-4c44-856a-7a43b9c756c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:54.895757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cf2c7-2f9e-4c44-856a-7a43b9c756c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:54.898728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cf2c7-2f9e-4c44-856a-7a43b9c756c6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-880cf2c7-2f9e-4c44-856a-7a43b9c756c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:44:54.934339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:54.935564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:54.937210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:44:54.937210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:44:54.937210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:54.937210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:54.937210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 622/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:44:54.943312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd49c50-888a-4e46-99f6-c9e16c5d39fa None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:55.048303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:55.056072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd49c50-888a-4e46-99f6-c9e16c5d39fa None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:55.094026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc24b2a-3131-46ad-9e31-2d9d70c981b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:55.094722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc24b2a-3131-46ad-9e31-2d9d70c981b1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:55.095011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc24b2a-3131-46ad-9e31-2d9d70c981b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:55.095290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc24b2a-3131-46ad-9e31-2d9d70c981b1 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:55.096031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc24b2a-3131-46ad-9e31-2d9d70c981b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:55.096339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc24b2a-3131-46ad-9e31-2d9d70c981b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:55.097751 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc24b2a-3131-46ad-9e31-2d9d70c981b1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:55.109759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 621/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:55.116168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cf2c7-2f9e-4c44-856a-7a43b9c756c6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:55.201346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:55.224139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cf2c7-2f9e-4c44-856a-7a43b9c756c6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:55.260714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834aa78-aa09-4e04-9d7e-2509ef10f115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:55.262254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834aa78-aa09-4e04-9d7e-2509ef10f115 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:55.262616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834aa78-aa09-4e04-9d7e-2509ef10f115 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:55.263190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834aa78-aa09-4e04-9d7e-2509ef10f115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:55.272228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834aa78-aa09-4e04-9d7e-2509ef10f115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:55.273732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834aa78-aa09-4e04-9d7e-2509ef10f115 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-a834aa78-aa09-4e04-9d7e-2509ef10f115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:44:55.288071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:55.290123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:55.291867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:44:55.291867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:44:55.291867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:44:55.291867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:44:55.291867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:44:55.291867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:44:55.291867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:44:55.291867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:55.291867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 623/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:44:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:44:55.295821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc24b2a-3131-46ad-9e31-2d9d70c981b1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:55.402110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc24b2a-3131-46ad-9e31-2d9d70c981b1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:55.438377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f83a0-2649-4fbf-9eb5-b498427d3578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:55.439038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f83a0-2649-4fbf-9eb5-b498427d3578 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:55.439322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f83a0-2649-4fbf-9eb5-b498427d3578 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:55.439604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f83a0-2649-4fbf-9eb5-b498427d3578 None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:55.440386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f83a0-2649-4fbf-9eb5-b498427d3578 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:55.440750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f83a0-2649-4fbf-9eb5-b498427d3578 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:55.442233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f83a0-2649-4fbf-9eb5-b498427d3578 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:55.454984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 622/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:44:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:55.461520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834aa78-aa09-4e04-9d7e-2509ef10f115 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:55.591132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834aa78-aa09-4e04-9d7e-2509ef10f115 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:55.633080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44ed01-c7e7-4f99-be36-021d1decf1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:55.633760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44ed01-c7e7-4f99-be36-021d1decf1bb None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:55.634046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44ed01-c7e7-4f99-be36-021d1decf1bb None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:55.634323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44ed01-c7e7-4f99-be36-021d1decf1bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:55.635365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44ed01-c7e7-4f99-be36-021d1decf1bb None admin] RBAC: Adding query filter params (role.id=65e4ab8e291442e3858317b0aa6c2d8a, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=1c1b23b83544423abc6919d805916b8d) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:55.635690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44ed01-c7e7-4f99-be36-021d1decf1bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:55.637198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44ed01-c7e7-4f99-be36-021d1decf1bb None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:55.644414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 624/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:44:55 2022] GET /identity/v3/role_assignments?user.id=1c1b23b83544423abc6919d805916b8d&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=65e4ab8e291442e3858317b0aa6c2d8a => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:57.856273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1ee8b-6758-4b21-aba7-36177c556d2b None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:57.856873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1ee8b-6758-4b21-aba7-36177c556d2b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:57.856990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1ee8b-6758-4b21-aba7-36177c556d2b None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:57.858956 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 623/1247] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:44:57.868288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00eb849-dc28-4d7e-8f96-c828c2c6406f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:57.868889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00eb849-dc28-4d7e-8f96-c828c2c6406f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:57.869317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00eb849-dc28-4d7e-8f96-c828c2c6406f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:57.878903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:57.898207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b00eb849-dc28-4d7e-8f96-c828c2c6406f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:44:57.900205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b00eb849-dc28-4d7e-8f96-c828c2c6406f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:44:57.993164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 625/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:44:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:44:58.017019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1ee8b-6758-4b21-aba7-36177c556d2b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:58.036841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:58.125707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1ee8b-6758-4b21-aba7-36177c556d2b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:58.162196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c588-89d1-49ba-997a-bb5ae0f51a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:58.163552 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c588-89d1-49ba-997a-bb5ae0f51a91 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:58.164600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c588-89d1-49ba-997a-bb5ae0f51a91 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:58.164966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c588-89d1-49ba-997a-bb5ae0f51a91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:58.176882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188c588-89d1-49ba-997a-bb5ae0f51a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-1188c588-89d1-49ba-997a-bb5ae0f51a91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:44:58.184617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:58.186301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:44:58.187876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 624/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:44:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:58.192179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00eb849-dc28-4d7e-8f96-c828c2c6406f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:58.305276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00eb849-dc28-4d7e-8f96-c828c2c6406f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:58.338358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:58.344454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e0d79-fc95-4564-ada3-9970bc57eeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:58.345235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e0d79-fc95-4564-ada3-9970bc57eeb6 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:58.345564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e0d79-fc95-4564-ada3-9970bc57eeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:58.345882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e0d79-fc95-4564-ada3-9970bc57eeb6 None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:58.346741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e0d79-fc95-4564-ada3-9970bc57eeb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:58.347094 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e0d79-fc95-4564-ada3-9970bc57eeb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:58.348531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e0d79-fc95-4564-ada3-9970bc57eeb6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:58.362135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 626/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:44:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:58.367080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c588-89d1-49ba-997a-bb5ae0f51a91 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:58.467664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:58.493630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c588-89d1-49ba-997a-bb5ae0f51a91 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:58.532917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11b1e7-0055-436e-9eec-d2fff71d39a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:58.533726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11b1e7-0055-436e-9eec-d2fff71d39a1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:58.534010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11b1e7-0055-436e-9eec-d2fff71d39a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:58.534285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11b1e7-0055-436e-9eec-d2fff71d39a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:58.546819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11b1e7-0055-436e-9eec-d2fff71d39a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:58.548309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11b1e7-0055-436e-9eec-d2fff71d39a1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-0b11b1e7-0055-436e-9eec-d2fff71d39a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:44:58.568297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:58.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:58.571873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:44:58.571873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:44:58.571873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:58.571873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:44:58.571873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 625/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:44:58.577948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e0d79-fc95-4564-ada3-9970bc57eeb6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:58.693080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e0d79-fc95-4564-ada3-9970bc57eeb6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:58.732435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e7ce3-47b0-4209-b081-0ba11ec6fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:58.733216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e7ce3-47b0-4209-b081-0ba11ec6fe2d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:58.733621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e7ce3-47b0-4209-b081-0ba11ec6fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:58.734444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e7ce3-47b0-4209-b081-0ba11ec6fe2d None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:58.735252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e7ce3-47b0-4209-b081-0ba11ec6fe2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:58.735664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e7ce3-47b0-4209-b081-0ba11ec6fe2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:58.737338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e7ce3-47b0-4209-b081-0ba11ec6fe2d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:58.751653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 627/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:58.757729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11b1e7-0055-436e-9eec-d2fff71d39a1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:58.873216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11b1e7-0055-436e-9eec-d2fff71d39a1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:58.903608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:58.912025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841949ca-e7d1-4193-ac46-1fdbbf6c1694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:58.912872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841949ca-e7d1-4193-ac46-1fdbbf6c1694 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:58.913427 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841949ca-e7d1-4193-ac46-1fdbbf6c1694 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:58.913706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841949ca-e7d1-4193-ac46-1fdbbf6c1694 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:58.922476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841949ca-e7d1-4193-ac46-1fdbbf6c1694 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:58.923988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841949ca-e7d1-4193-ac46-1fdbbf6c1694 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-841949ca-e7d1-4193-ac46-1fdbbf6c1694 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:44:58.941626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:58.942863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:44:58.945562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 626/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:44:58.952807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e7ce3-47b0-4209-b081-0ba11ec6fe2d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:59.023359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:59.071481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e7ce3-47b0-4209-b081-0ba11ec6fe2d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:59.108135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12887c-0f95-4f50-a512-7d7b831f0e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:59.109487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12887c-0f95-4f50-a512-7d7b831f0e63 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:59.109772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12887c-0f95-4f50-a512-7d7b831f0e63 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:59.110050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12887c-0f95-4f50-a512-7d7b831f0e63 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:59.110912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12887c-0f95-4f50-a512-7d7b831f0e63 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:59.111225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12887c-0f95-4f50-a512-7d7b831f0e63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:59.112711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12887c-0f95-4f50-a512-7d7b831f0e63 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:59.141043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 628/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Jul 24 19:44:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:44:59.147451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841949ca-e7d1-4193-ac46-1fdbbf6c1694 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:59.257458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841949ca-e7d1-4193-ac46-1fdbbf6c1694 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:59.296608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716825f6-ec5c-46d4-ba24-e9d0bac2dc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:59.297744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716825f6-ec5c-46d4-ba24-e9d0bac2dc8b None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:59.298105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716825f6-ec5c-46d4-ba24-e9d0bac2dc8b None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:59.298459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716825f6-ec5c-46d4-ba24-e9d0bac2dc8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:59.383755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:59.398551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716825f6-ec5c-46d4-ba24-e9d0bac2dc8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:59.400207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716825f6-ec5c-46d4-ba24-e9d0bac2dc8b None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:59.516323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 627/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:44:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 24 19:44:59.522687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12887c-0f95-4f50-a512-7d7b831f0e63 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:59.637510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12887c-0f95-4f50-a512-7d7b831f0e63 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:59.653603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:59.678010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ea000-b7dd-429c-97a7-2dab1848a6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:59.678828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ea000-b7dd-429c-97a7-2dab1848a6ba None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:59.679325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ea000-b7dd-429c-97a7-2dab1848a6ba None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:59.679605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ea000-b7dd-429c-97a7-2dab1848a6ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:59.691222 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ea000-b7dd-429c-97a7-2dab1848a6ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:59.693509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ea000-b7dd-429c-97a7-2dab1848a6ba None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-d42ea000-b7dd-429c-97a7-2dab1848a6ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:44:59.714071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:44:59.715767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:44:59.718295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:44:59.718295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:44:59.718295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:44:59.718295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:44:59.718295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 629/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:44:59.724017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716825f6-ec5c-46d4-ba24-e9d0bac2dc8b None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:44:59.849555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:44:59.863043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716825f6-ec5c-46d4-ba24-e9d0bac2dc8b None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:44:59.901281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf7caf-935b-4b1f-959c-2df475125ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:44:59.902099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf7caf-935b-4b1f-959c-2df475125ac7 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:44:59.902232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf7caf-935b-4b1f-959c-2df475125ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:44:59.902510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf7caf-935b-4b1f-959c-2df475125ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:44:59.903257 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf7caf-935b-4b1f-959c-2df475125ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:44:59.903595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf7caf-935b-4b1f-959c-2df475125ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:44:59.904998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf7caf-935b-4b1f-959c-2df475125ac7 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:44:59.919178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 628/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:44:59.927963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ea000-b7dd-429c-97a7-2dab1848a6ba None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:00.041972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ea000-b7dd-429c-97a7-2dab1848a6ba None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:00.079765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee650a-3f21-4674-b5d1-6b5a1820f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:00.080579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee650a-3f21-4674-b5d1-6b5a1820f655 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:00.080891 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee650a-3f21-4674-b5d1-6b5a1820f655 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:00.081171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee650a-3f21-4674-b5d1-6b5a1820f655 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:00.088747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee650a-3f21-4674-b5d1-6b5a1820f655 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:00.090262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee650a-3f21-4674-b5d1-6b5a1820f655 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-87ee650a-3f21-4674-b5d1-6b5a1820f655 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:45:00.112339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:00.114228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:00.116174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:45:00.116174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:45:00.116174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:45:00.116174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:45:00.116174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:45:00.116174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:45:00.116174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:45:00.116174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:00.116174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 630/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:00.121784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf7caf-935b-4b1f-959c-2df475125ac7 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:00.238684 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf7caf-935b-4b1f-959c-2df475125ac7 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:00.277298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7364b-46da-448b-9596-cda44024729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:00.277991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7364b-46da-448b-9596-cda44024729c None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:00.278412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7364b-46da-448b-9596-cda44024729c None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:00.278688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7364b-46da-448b-9596-cda44024729c None admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:00.279491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7364b-46da-448b-9596-cda44024729c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:00.279802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7364b-46da-448b-9596-cda44024729c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:00.281395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7364b-46da-448b-9596-cda44024729c None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:00.296353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 629/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:45:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:00.303266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee650a-3f21-4674-b5d1-6b5a1820f655 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:00.334086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:00.416958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee650a-3f21-4674-b5d1-6b5a1820f655 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:00.460918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dab14-d923-49d2-8d82-97b328346fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:00.461807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dab14-d923-49d2-8d82-97b328346fa5 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:00.462100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dab14-d923-49d2-8d82-97b328346fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:00.462375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dab14-d923-49d2-8d82-97b328346fa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:00.463245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dab14-d923-49d2-8d82-97b328346fa5 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=1c1b23b83544423abc6919d805916b8d) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:00.463561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dab14-d923-49d2-8d82-97b328346fa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:00.465090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dab14-d923-49d2-8d82-97b328346fa5 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:00.472033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 631/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:45:00 2022] GET /identity/v3/role_assignments?user.id=1c1b23b83544423abc6919d805916b8d&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:02.634582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866b73a-eea5-4cc6-9b9b-92ae49c4160e None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:02.635181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866b73a-eea5-4cc6-9b9b-92ae49c4160e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:02.635181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866b73a-eea5-4cc6-9b9b-92ae49c4160e None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:02.636704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 630/1261] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:45:02.643301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8bda1-484b-4a35-ac7f-ad18fbaa79bd None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:02.643864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8bda1-484b-4a35-ac7f-ad18fbaa79bd None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:02.643997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8bda1-484b-4a35-ac7f-ad18fbaa79bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:02.663707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-5bf8bda1-484b-4a35-ac7f-ad18fbaa79bd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:45:02.665177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5bf8bda1-484b-4a35-ac7f-ad18fbaa79bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:45:02.752369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 632/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:45:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:45:02.776606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866b73a-eea5-4cc6-9b9b-92ae49c4160e None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:02.789895 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:02.897175 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866b73a-eea5-4cc6-9b9b-92ae49c4160e None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:02.939519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571140df-b9d8-4a81-a9d7-d163555c0fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:02.940431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571140df-b9d8-4a81-a9d7-d163555c0fb4 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:02.940832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571140df-b9d8-4a81-a9d7-d163555c0fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:02.941160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571140df-b9d8-4a81-a9d7-d163555c0fb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:02.953918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571140df-b9d8-4a81-a9d7-d163555c0fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-571140df-b9d8-4a81-a9d7-d163555c0fb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:45:02.962619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:02.964731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:45:02.966703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 631/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:45:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:02.971861 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8bda1-484b-4a35-ac7f-ad18fbaa79bd None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:02.992671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:03.086198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8bda1-484b-4a35-ac7f-ad18fbaa79bd None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:03.134948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d4979-44a8-4fbb-a59b-ae8745e43e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:03.135791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d4979-44a8-4fbb-a59b-ae8745e43e9e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:03.136189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d4979-44a8-4fbb-a59b-ae8745e43e9e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:03.136601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d4979-44a8-4fbb-a59b-ae8745e43e9e None admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:03.137470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d4979-44a8-4fbb-a59b-ae8745e43e9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:03.137840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d4979-44a8-4fbb-a59b-ae8745e43e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:03.139275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d4979-44a8-4fbb-a59b-ae8745e43e9e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:03.149256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 633/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:45:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:03.158717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571140df-b9d8-4a81-a9d7-d163555c0fb4 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:03.234899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:03.271848 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571140df-b9d8-4a81-a9d7-d163555c0fb4 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:03.313997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d7408-e4fb-4364-bfc8-f8a18cfee528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:03.314965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d7408-e4fb-4364-bfc8-f8a18cfee528 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:03.315472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d7408-e4fb-4364-bfc8-f8a18cfee528 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:03.315803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d7408-e4fb-4364-bfc8-f8a18cfee528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:03.327924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d7408-e4fb-4364-bfc8-f8a18cfee528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:03.329501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d7408-e4fb-4364-bfc8-f8a18cfee528 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-b82d7408-e4fb-4364-bfc8-f8a18cfee528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:45:03.348628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:03.350438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:03.352155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:45:03.352155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:45:03.352155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:45:03.352155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:45:03.352155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 632/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:03.356244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d4979-44a8-4fbb-a59b-ae8745e43e9e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:03.456195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:03.486813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d4979-44a8-4fbb-a59b-ae8745e43e9e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:03.527168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd83a33-7aba-4687-a8f0-51f028c58a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:03.527854 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd83a33-7aba-4687-a8f0-51f028c58a5d None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:03.528140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd83a33-7aba-4687-a8f0-51f028c58a5d None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:03.528418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd83a33-7aba-4687-a8f0-51f028c58a5d None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:03.529196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd83a33-7aba-4687-a8f0-51f028c58a5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:03.529508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd83a33-7aba-4687-a8f0-51f028c58a5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:03.531589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd83a33-7aba-4687-a8f0-51f028c58a5d None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:03.547346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 634/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:03.553959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d7408-e4fb-4364-bfc8-f8a18cfee528 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:03.671205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d7408-e4fb-4364-bfc8-f8a18cfee528 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:03.699844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:03.718211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dfd6e-40d6-49f1-9f85-eb34a2f5da42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:03.719322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dfd6e-40d6-49f1-9f85-eb34a2f5da42 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:03.719609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dfd6e-40d6-49f1-9f85-eb34a2f5da42 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:03.719893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dfd6e-40d6-49f1-9f85-eb34a2f5da42 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:03.728576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318dfd6e-40d6-49f1-9f85-eb34a2f5da42 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:03.730091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318dfd6e-40d6-49f1-9f85-eb34a2f5da42 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-318dfd6e-40d6-49f1-9f85-eb34a2f5da42 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:45:03.745859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:03.750638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:45:03.752892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 633/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:45:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:45:03.754325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd83a33-7aba-4687-a8f0-51f028c58a5d None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:03.869162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd83a33-7aba-4687-a8f0-51f028c58a5d None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:03.913468 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9c86-1de9-42b4-a43c-1a8e5f4bb8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:03.914517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9c86-1de9-42b4-a43c-1a8e5f4bb8c7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:03.914901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9c86-1de9-42b4-a43c-1a8e5f4bb8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:03.915360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9c86-1de9-42b4-a43c-1a8e5f4bb8c7 None admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:03.916991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae9c86-1de9-42b4-a43c-1a8e5f4bb8c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:03.917586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae9c86-1de9-42b4-a43c-1a8e5f4bb8c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:03.920374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae9c86-1de9-42b4-a43c-1a8e5f4bb8c7 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:03.933860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:03.943769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 635/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Jul 24 19:45:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:03.949933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dfd6e-40d6-49f1-9f85-eb34a2f5da42 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:04.083870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dfd6e-40d6-49f1-9f85-eb34a2f5da42 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:04.129151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e36a8-a376-445b-9859-168432fe12ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:04.129974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e36a8-a376-445b-9859-168432fe12ef None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:04.130263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e36a8-a376-445b-9859-168432fe12ef None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:04.130541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e36a8-a376-445b-9859-168432fe12ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:04.143792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e36a8-a376-445b-9859-168432fe12ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:04.145311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e36a8-a376-445b-9859-168432fe12ef None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-968e36a8-a376-445b-9859-168432fe12ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:45:04.164703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:04.166489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:04.168158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:45:04.168158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:45:04.168158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:45:04.168158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:45:04.168158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 634/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:04.173541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9c86-1de9-42b4-a43c-1a8e5f4bb8c7 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:04.294681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9c86-1de9-42b4-a43c-1a8e5f4bb8c7 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:04.337573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d82380-2e25-4eb5-9c84-6fb6015e96f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:04.338680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d82380-2e25-4eb5-9c84-6fb6015e96f6 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:04.339154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d82380-2e25-4eb5-9c84-6fb6015e96f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:04.339591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d82380-2e25-4eb5-9c84-6fb6015e96f6 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:04.341021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d82380-2e25-4eb5-9c84-6fb6015e96f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:04.341554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d82380-2e25-4eb5-9c84-6fb6015e96f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:04.344202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d82380-2e25-4eb5-9c84-6fb6015e96f6 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:04.359813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 636/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:04.369697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e36a8-a376-445b-9859-168432fe12ef None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:04.411843 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:04.502925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e36a8-a376-445b-9859-168432fe12ef None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:04.541600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14c3d2-bff4-4741-85a2-19ed721a849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:04.542415 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14c3d2-bff4-4741-85a2-19ed721a849a None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:04.542808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14c3d2-bff4-4741-85a2-19ed721a849a None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:04.543084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14c3d2-bff4-4741-85a2-19ed721a849a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:04.551653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14c3d2-bff4-4741-85a2-19ed721a849a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:04.553141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14c3d2-bff4-4741-85a2-19ed721a849a None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-cd14c3d2-bff4-4741-85a2-19ed721a849a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:45:04.569301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:04.570574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:04.571782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:45:04.571782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:45:04.571782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:45:04.571782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:45:04.571782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:45:04.571782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:45:04.571782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:45:04.571782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:45:04.571782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 635/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:45:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:04.580364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d82380-2e25-4eb5-9c84-6fb6015e96f6 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:04.674928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:04.725884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d82380-2e25-4eb5-9c84-6fb6015e96f6 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:04.769939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05811c99-d61b-465d-a33f-7568e4550d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:04.770735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05811c99-d61b-465d-a33f-7568e4550d7c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:04.770906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05811c99-d61b-465d-a33f-7568e4550d7c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:04.771189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05811c99-d61b-465d-a33f-7568e4550d7c None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:04.772039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05811c99-d61b-465d-a33f-7568e4550d7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:04.772354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05811c99-d61b-465d-a33f-7568e4550d7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:04.773897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05811c99-d61b-465d-a33f-7568e4550d7c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:04.789655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 637/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:45:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:04.796570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14c3d2-bff4-4741-85a2-19ed721a849a None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:04.914005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:04.926917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14c3d2-bff4-4741-85a2-19ed721a849a None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:04.980918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b0f0f-caf6-44ad-bd5d-607b112c379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:04.982207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b0f0f-caf6-44ad-bd5d-607b112c379d None admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:04.982620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b0f0f-caf6-44ad-bd5d-607b112c379d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:04.983099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b0f0f-caf6-44ad-bd5d-607b112c379d None admin] PATH_INFO: `/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/1c1b23b83544423abc6919d805916b8d/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:04.986141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b0f0f-caf6-44ad-bd5d-607b112c379d None admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9dcb1ea40484080c0aeb065c702e9, user_id=1c1b23b83544423abc6919d805916b8d, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:04.987610 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b0f0f-caf6-44ad-bd5d-607b112c379d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:05.101190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 636/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Jul 24 19:45:04 2022] PUT /identity/v3/projects/1ee9dcb1ea40484080c0aeb065c702e9/users/1c1b23b83544423abc6919d805916b8d/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:45:07.261599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab207451-3b08-40c1-9b1f-246d46fece7c None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:07.262199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab207451-3b08-40c1-9b1f-246d46fece7c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:07.262199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab207451-3b08-40c1-9b1f-246d46fece7c None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:07.263584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 638/1274] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:45:07.269993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30764a-6c47-40d4-b3ec-be992b74534a None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:07.270556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30764a-6c47-40d4-b3ec-be992b74534a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:07.270726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30764a-6c47-40d4-b3ec-be992b74534a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:07.288715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-df30764a-6c47-40d4-b3ec-be992b74534a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:45:07.289954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-df30764a-6c47-40d4-b3ec-be992b74534a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:45:07.371819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 637/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:45:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:45:07.396148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab207451-3b08-40c1-9b1f-246d46fece7c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:07.512876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab207451-3b08-40c1-9b1f-246d46fece7c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:07.532991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:07.560353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5544ff-e56f-4f0d-9e33-48e5d23a67f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:07.562272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5544ff-e56f-4f0d-9e33-48e5d23a67f8 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:07.562666 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5544ff-e56f-4f0d-9e33-48e5d23a67f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:07.563161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5544ff-e56f-4f0d-9e33-48e5d23a67f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:07.583257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5544ff-e56f-4f0d-9e33-48e5d23a67f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-cb5544ff-e56f-4f0d-9e33-48e5d23a67f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 24 19:45:07.598338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:07.599548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 24 19:45:07.600494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:07.601420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 639/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Jul 24 19:45:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:07.607680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30764a-6c47-40d4-b3ec-be992b74534a None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:07.727098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:07.750183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30764a-6c47-40d4-b3ec-be992b74534a None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:07.798199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22aca2-1699-46d6-999d-d1acb2ca3154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:07.798926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22aca2-1699-46d6-999d-d1acb2ca3154 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:07.799258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22aca2-1699-46d6-999d-d1acb2ca3154 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:07.799576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22aca2-1699-46d6-999d-d1acb2ca3154 None admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:07.800411 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22aca2-1699-46d6-999d-d1acb2ca3154 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:07.800813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22aca2-1699-46d6-999d-d1acb2ca3154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:07.802765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22aca2-1699-46d6-999d-d1acb2ca3154 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:07.813515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 638/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Jul 24 19:45:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:07.823520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5544ff-e56f-4f0d-9e33-48e5d23a67f8 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:07.944395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5544ff-e56f-4f0d-9e33-48e5d23a67f8 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:07.987198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e92e-f360-48ff-a216-44e086639a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:07.988007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e92e-f360-48ff-a216-44e086639a7e None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:07.988287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e92e-f360-48ff-a216-44e086639a7e None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:07.988635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e92e-f360-48ff-a216-44e086639a7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:08.002413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3e92e-f360-48ff-a216-44e086639a7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:08.005717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3e92e-f360-48ff-a216-44e086639a7e None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-abd3e92e-f360-48ff-a216-44e086639a7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:45:08.027654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:08.029039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:08.031410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:45:08.031410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:45:08.031410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:45:08.031410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:08.031410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 640/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:08.035656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22aca2-1699-46d6-999d-d1acb2ca3154 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:08.168463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22aca2-1699-46d6-999d-d1acb2ca3154 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:08.229273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5681917-40d2-41f6-83d5-b9761eb19a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:08.230449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5681917-40d2-41f6-83d5-b9761eb19a1d None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:08.230963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5681917-40d2-41f6-83d5-b9761eb19a1d None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:08.231452 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5681917-40d2-41f6-83d5-b9761eb19a1d None admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:08.232890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5681917-40d2-41f6-83d5-b9761eb19a1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:08.233444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5681917-40d2-41f6-83d5-b9761eb19a1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:08.236084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5681917-40d2-41f6-83d5-b9761eb19a1d None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:08.259618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 639/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:08.266067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e92e-f360-48ff-a216-44e086639a7e None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:08.305837 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:08.394583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e92e-f360-48ff-a216-44e086639a7e None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:08.449391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d46d2-092b-4dff-8058-c6626825940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:08.450354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d46d2-092b-4dff-8058-c6626825940f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:08.450823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d46d2-092b-4dff-8058-c6626825940f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:08.451353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d46d2-092b-4dff-8058-c6626825940f None admin] PATH_INFO: `/v3/users/placement` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:08.462494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d46d2-092b-4dff-8058-c6626825940f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:08.464680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d46d2-092b-4dff-8058-c6626825940f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-770d46d2-092b-4dff-8058-c6626825940f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 24 19:45:08.482471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:08.484709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:08.486454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 641/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:45:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:08.494900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5681917-40d2-41f6-83d5-b9761eb19a1d None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:08.502027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:08.676325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5681917-40d2-41f6-83d5-b9761eb19a1d None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:08.723708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd11a4-2566-4952-99cc-d19bd7ff70b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:08.724442 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd11a4-2566-4952-99cc-d19bd7ff70b1 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:08.724835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd11a4-2566-4952-99cc-d19bd7ff70b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:08.725288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd11a4-2566-4952-99cc-d19bd7ff70b1 None admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:08.726315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd11a4-2566-4952-99cc-d19bd7ff70b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:08.726757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd11a4-2566-4952-99cc-d19bd7ff70b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:08.728390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd11a4-2566-4952-99cc-d19bd7ff70b1 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:08.751314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 640/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Jul 24 19:45:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:45:08.757740 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d46d2-092b-4dff-8058-c6626825940f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:08.903664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:08.916757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d46d2-092b-4dff-8058-c6626825940f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:08.968208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9a6b-7c51-4599-b796-b0aef7f7335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:08.969036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9a6b-7c51-4599-b796-b0aef7f7335f None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:08.969384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9a6b-7c51-4599-b796-b0aef7f7335f None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:08.969736 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9a6b-7c51-4599-b796-b0aef7f7335f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:09.046517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e9a6b-7c51-4599-b796-b0aef7f7335f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:09.048064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e9a6b-7c51-4599-b796-b0aef7f7335f None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:09.179768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:09.199346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 642/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Jul 24 19:45:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 24 19:45:09.207870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd11a4-2566-4952-99cc-d19bd7ff70b1 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:09.293992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:09.335139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd11a4-2566-4952-99cc-d19bd7ff70b1 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:09.373083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102ddf3-c7fc-42df-81e0-1302f0b4b1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:09.373914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102ddf3-c7fc-42df-81e0-1302f0b4b1ae None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:09.374179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102ddf3-c7fc-42df-81e0-1302f0b4b1ae None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:09.374458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102ddf3-c7fc-42df-81e0-1302f0b4b1ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:09.385432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102ddf3-c7fc-42df-81e0-1302f0b4b1ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:09.386903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102ddf3-c7fc-42df-81e0-1302f0b4b1ae None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-5102ddf3-c7fc-42df-81e0-1302f0b4b1ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 24 19:45:09.406360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:09.408723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:09.411056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 24 19:45:09.411056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 24 19:45:09.411056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 24 19:45:09.411056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:45:09.411056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 641/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 24 19:45:09.414852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9a6b-7c51-4599-b796-b0aef7f7335f None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:09.539972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9a6b-7c51-4599-b796-b0aef7f7335f None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:09.599077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ee991-1d99-4810-ad3c-af293cc55619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:09.600167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ee991-1d99-4810-ad3c-af293cc55619 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:09.600692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ee991-1d99-4810-ad3c-af293cc55619 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:09.601175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ee991-1d99-4810-ad3c-af293cc55619 None admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:09.602564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ee991-1d99-4810-ad3c-af293cc55619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:09.603125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ee991-1d99-4810-ad3c-af293cc55619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:09.605784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ee991-1d99-4810-ad3c-af293cc55619 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:09.629005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 643/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Jul 24 19:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:09.634823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102ddf3-c7fc-42df-81e0-1302f0b4b1ae None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:09.755422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102ddf3-c7fc-42df-81e0-1302f0b4b1ae None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:09.775031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:09.800181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb54df-c374-4302-87ff-4b3ad2bfb3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:09.801219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb54df-c374-4302-87ff-4b3ad2bfb3d0 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:09.801424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb54df-c374-4302-87ff-4b3ad2bfb3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:09.801699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb54df-c374-4302-87ff-4b3ad2bfb3d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:09.809794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb54df-c374-4302-87ff-4b3ad2bfb3d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:09.812081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb54df-c374-4302-87ff-4b3ad2bfb3d0 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application [None req-eeeb54df-c374-4302-87ff-4b3ad2bfb3d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 24 19:45:09.828804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:09.835954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:09.837847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 24 19:45:09.837847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 24 19:45:09.837847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 24 19:45:09.837847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 24 19:45:09.837847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 24 19:45:09.837847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 24 19:45:09.837847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 24 19:45:09.837847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: ERROR keystone.server.flask.application Jul 24 19:45:09.837847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 642/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Jul 24 19:45:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:09.838849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ee991-1d99-4810-ad3c-af293cc55619 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:09.896565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:09.953370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ee991-1d99-4810-ad3c-af293cc55619 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:09.992951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e43447-f12f-4f8f-b1f1-2f3a9b019e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:09.993713 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e43447-f12f-4f8f-b1f1-2f3a9b019e17 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:09.994043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e43447-f12f-4f8f-b1f1-2f3a9b019e17 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:09.994395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e43447-f12f-4f8f-b1f1-2f3a9b019e17 None admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:09.995311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e43447-f12f-4f8f-b1f1-2f3a9b019e17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:09.995670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e43447-f12f-4f8f-b1f1-2f3a9b019e17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:09.997294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e43447-f12f-4f8f-b1f1-2f3a9b019e17 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:10.011142 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 644/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Jul 24 19:45:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:45:10.018682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb54df-c374-4302-87ff-4b3ad2bfb3d0 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:10.150183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb54df-c374-4302-87ff-4b3ad2bfb3d0 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:10.193840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839185b-e4f0-47e3-ad3d-9e7642d588f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:10.194898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839185b-e4f0-47e3-ad3d-9e7642d588f3 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:10.195141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839185b-e4f0-47e3-ad3d-9e7642d588f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:10.195549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839185b-e4f0-47e3-ad3d-9e7642d588f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:10.196387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839185b-e4f0-47e3-ad3d-9e7642d588f3 None admin] RBAC: Adding query filter params (role.id=c688b8fdcef8429f85cb90f675f24ca8, scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9, user.id=1c1b23b83544423abc6919d805916b8d) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:10.196791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839185b-e4f0-47e3-ad3d-9e7642d588f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:10.198375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839185b-e4f0-47e3-ad3d-9e7642d588f3 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:10.205750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 643/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Jul 24 19:45:10 2022] GET /identity/v3/role_assignments?user.id=1c1b23b83544423abc6919d805916b8d&scope.project.id=1ee9dcb1ea40484080c0aeb065c702e9&role.id=c688b8fdcef8429f85cb90f675f24ca8 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 19:45:12.380584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590daefe-ee59-4059-9250-50c26ddaaede None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:12.381209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590daefe-ee59-4059-9250-50c26ddaaede None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:12.381367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590daefe-ee59-4059-9250-50c26ddaaede None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:12.382768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 645/1288] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:45:12.388483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa4d4f-511c-4666-af06-791c8da8e819 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:12.392628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa4d4f-511c-4666-af06-791c8da8e819 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:12.392628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa4d4f-511c-4666-af06-791c8da8e819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:12.406814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-fafa4d4f-511c-4666-af06-791c8da8e819 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:45:12.408034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fafa4d4f-511c-4666-af06-791c8da8e819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:45:12.441789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:12.478793 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 644/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:45:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:45:12.502114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590daefe-ee59-4059-9250-50c26ddaaede None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:12.608235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590daefe-ee59-4059-9250-50c26ddaaede None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:12.619178 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:12.642666 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b277e7-896e-4afb-8bd9-d93782422eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:12.643380 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b277e7-896e-4afb-8bd9-d93782422eb7 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:12.643708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b277e7-896e-4afb-8bd9-d93782422eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:12.643985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b277e7-896e-4afb-8bd9-d93782422eb7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:12.644636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b277e7-896e-4afb-8bd9-d93782422eb7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:12.646031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b277e7-896e-4afb-8bd9-d93782422eb7 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-55b277e7-896e-4afb-8bd9-d93782422eb7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:12.660154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:12.662089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:12.663873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 646/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:45:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:12.671902 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa4d4f-511c-4666-af06-791c8da8e819 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:12.775563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa4d4f-511c-4666-af06-791c8da8e819 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:12.812092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a0189-5397-4086-8744-759098180ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:12.812815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a0189-5397-4086-8744-759098180ee6 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:12.813100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a0189-5397-4086-8744-759098180ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:12.813378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a0189-5397-4086-8744-759098180ee6 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:12.814099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a0189-5397-4086-8744-759098180ee6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:12.814435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a0189-5397-4086-8744-759098180ee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:12.815823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a0189-5397-4086-8744-759098180ee6 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:12.821363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 645/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Sun Jul 24 19:45:12 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:12.830292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b277e7-896e-4afb-8bd9-d93782422eb7 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:12.950171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b277e7-896e-4afb-8bd9-d93782422eb7 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:13.000619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb214417-763f-4ef5-9f41-92c00b6d2d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:13.001312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214417-763f-4ef5-9f41-92c00b6d2d8c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:13.001596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214417-763f-4ef5-9f41-92c00b6d2d8c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:13.001869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214417-763f-4ef5-9f41-92c00b6d2d8c None admin] PATH_INFO: `/v3/services` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:13.002578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb214417-763f-4ef5-9f41-92c00b6d2d8c None admin] RBAC: Adding query filter params (type=placement) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:13.002884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb214417-763f-4ef5-9f41-92c00b6d2d8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:13.004323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb214417-763f-4ef5-9f41-92c00b6d2d8c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:13.009414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 647/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Sun Jul 24 19:45:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:15.115286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0291a-b4a8-4604-aa85-31d4cb666abc None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:15.115929 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0291a-b4a8-4604-aa85-31d4cb666abc None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:15.115929 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0291a-b4a8-4604-aa85-31d4cb666abc None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:15.117387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 646/1293] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:45:15.123214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b272b6-21c6-4573-a8f8-62c907cb6cf2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:15.124157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b272b6-21c6-4573-a8f8-62c907cb6cf2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:15.124157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b272b6-21c6-4573-a8f8-62c907cb6cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:15.142833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-63b272b6-21c6-4573-a8f8-62c907cb6cf2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:45:15.144052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-63b272b6-21c6-4573-a8f8-62c907cb6cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:45:15.184297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:15.228438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 648/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:45:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:45:15.271563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0291a-b4a8-4604-aa85-31d4cb666abc None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:15.305556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:15.384308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0291a-b4a8-4604-aa85-31d4cb666abc None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:15.424765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5633bb-4e44-4aa8-9d4b-a67984fa1e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:15.425607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5633bb-4e44-4aa8-9d4b-a67984fa1e90 None admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:15.426670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5633bb-4e44-4aa8-9d4b-a67984fa1e90 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:15.426670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5633bb-4e44-4aa8-9d4b-a67984fa1e90 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:15.427587 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5633bb-4e44-4aa8-9d4b-a67984fa1e90 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:15.429203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5633bb-4e44-4aa8-9d4b-a67984fa1e90 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:15.527984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 647/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Jul 24 19:45:15 2022] POST /identity/v3/services => generated 246 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:45:17.676590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517d6db-c514-4b31-bfa0-fb1a344045f8 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:17.677168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517d6db-c514-4b31-bfa0-fb1a344045f8 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:17.677542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517d6db-c514-4b31-bfa0-fb1a344045f8 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:17.680194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 649/1296] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:45:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:45:17.688147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c20dd4-41fe-41e4-ba6d-71cde9cb5fb3 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:17.688635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c20dd4-41fe-41e4-ba6d-71cde9cb5fb3 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:17.688806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c20dd4-41fe-41e4-ba6d-71cde9cb5fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:17.705621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-66c20dd4-41fe-41e4-ba6d-71cde9cb5fb3 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:45:17.706942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-66c20dd4-41fe-41e4-ba6d-71cde9cb5fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:45:17.789294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 648/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:45:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 24 19:45:17.812360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517d6db-c514-4b31-bfa0-fb1a344045f8 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:17.930037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517d6db-c514-4b31-bfa0-fb1a344045f8 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:17.977754 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c248-8eed-4515-90c8-862d0e27d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:17.978785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c248-8eed-4515-90c8-862d0e27d3ad None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:17.979168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c248-8eed-4515-90c8-862d0e27d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:17.979539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c248-8eed-4515-90c8-862d0e27d3ad None admin] PATH_INFO: `/v3/services/placement` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:17.980294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10c248-8eed-4515-90c8-862d0e27d3ad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:17.981818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10c248-8eed-4515-90c8-862d0e27d3ad None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-af10c248-8eed-4515-90c8-862d0e27d3ad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:17.994728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:17.996953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:17.998521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 650/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:45:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:18.005104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c20dd4-41fe-41e4-ba6d-71cde9cb5fb3 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:18.037271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:18.126543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c20dd4-41fe-41e4-ba6d-71cde9cb5fb3 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:18.165564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456cd1c-3dcf-40ef-b3bd-f3b75e40aaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:18.166374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456cd1c-3dcf-40ef-b3bd-f3b75e40aaa2 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:18.166704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456cd1c-3dcf-40ef-b3bd-f3b75e40aaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:18.167037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456cd1c-3dcf-40ef-b3bd-f3b75e40aaa2 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:18.167845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456cd1c-3dcf-40ef-b3bd-f3b75e40aaa2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:18.168205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456cd1c-3dcf-40ef-b3bd-f3b75e40aaa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:18.169720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456cd1c-3dcf-40ef-b3bd-f3b75e40aaa2 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:18.175680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 649/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Sun Jul 24 19:45:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:18.186403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c248-8eed-4515-90c8-862d0e27d3ad None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:18.219801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:18.293366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c248-8eed-4515-90c8-862d0e27d3ad None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:18.331972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78a3ca-ce43-4d86-b9bc-7cb328e9437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:18.332637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78a3ca-ce43-4d86-b9bc-7cb328e9437c None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:18.332955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78a3ca-ce43-4d86-b9bc-7cb328e9437c None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:18.333228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78a3ca-ce43-4d86-b9bc-7cb328e9437c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:18.334025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78a3ca-ce43-4d86-b9bc-7cb328e9437c None admin] RBAC: Adding query filter params (interface=public, service_id=34ab376bcc7f43bd9afa1bf48ddec53a, region_id=RegionOne) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:18.334341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78a3ca-ce43-4d86-b9bc-7cb328e9437c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:18.335714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78a3ca-ce43-4d86-b9bc-7cb328e9437c None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:18.341516 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 651/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Jul 24 19:45:18 2022] GET /identity/v3/endpoints?service_id=34ab376bcc7f43bd9afa1bf48ddec53a&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:18.347376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456cd1c-3dcf-40ef-b3bd-f3b75e40aaa2 None admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:18.441165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:18.461540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456cd1c-3dcf-40ef-b3bd-f3b75e40aaa2 None admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:18.499795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881a381-65dd-4b80-b06e-f1cef4e917c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:18.500546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881a381-65dd-4b80-b06e-f1cef4e917c4 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:18.500904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881a381-65dd-4b80-b06e-f1cef4e917c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:18.501226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881a381-65dd-4b80-b06e-f1cef4e917c4 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:18.501956 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881a381-65dd-4b80-b06e-f1cef4e917c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:18.503589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881a381-65dd-4b80-b06e-f1cef4e917c4 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:18.511283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 650/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Jul 24 19:45:18 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 19:45:20.703996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150497b-b1f2-40e2-bed8-c19f062f840a None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:20.704540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150497b-b1f2-40e2-bed8-c19f062f840a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:20.704540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150497b-b1f2-40e2-bed8-c19f062f840a None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:20.705888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 652/1302] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:45:20.714484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5629f93-2de6-460e-999b-40836e7e8786 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:20.714996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5629f93-2de6-460e-999b-40836e7e8786 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:20.715489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5629f93-2de6-460e-999b-40836e7e8786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:20.738169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f5629f93-2de6-460e-999b-40836e7e8786 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:45:20.745559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f5629f93-2de6-460e-999b-40836e7e8786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:45:20.814058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 651/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:45:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 24 19:45:20.837996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150497b-b1f2-40e2-bed8-c19f062f840a None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:20.941957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:20.949512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150497b-b1f2-40e2-bed8-c19f062f840a None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:20.983954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb6696-8ff0-43d6-85a0-e1919898ac05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:20.984794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb6696-8ff0-43d6-85a0-e1919898ac05 None admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:20.985165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb6696-8ff0-43d6-85a0-e1919898ac05 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:20.985437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb6696-8ff0-43d6-85a0-e1919898ac05 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:20.986099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb6696-8ff0-43d6-85a0-e1919898ac05 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:20.987578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb6696-8ff0-43d6-85a0-e1919898ac05 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application [None req-cbcb6696-8ff0-43d6-85a0-e1919898ac05 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 24 19:45:21.001564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 24 19:45:21.002869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.get_or_create( Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application with Lock( Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self._enter() Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application return self.creator() Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application created_value = creator( Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: ERROR keystone.server.flask.application Jul 24 19:45:21.003734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 653/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Jul 24 19:45:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 24 19:45:21.009097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5629f93-2de6-460e-999b-40836e7e8786 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:21.100437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:21.124562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5629f93-2de6-460e-999b-40836e7e8786 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:21.159750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb96723-b18d-47ce-b307-77b5ef55bf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:21.160421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb96723-b18d-47ce-b307-77b5ef55bf97 None admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:21.160736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb96723-b18d-47ce-b307-77b5ef55bf97 None admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:21.161016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb96723-b18d-47ce-b307-77b5ef55bf97 None admin] PATH_INFO: `/v3/services` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:21.161724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb96723-b18d-47ce-b307-77b5ef55bf97 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:45:21.162035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb96723-b18d-47ce-b307-77b5ef55bf97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:21.163673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb96723-b18d-47ce-b307-77b5ef55bf97 None admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:21.169634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 652/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Sun Jul 24 19:45:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:45:21.176199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb6696-8ff0-43d6-85a0-e1919898ac05 None admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:45:21.294663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb6696-8ff0-43d6-85a0-e1919898ac05 None admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:45:21.340555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350860f9-b4de-44c5-adb6-6c7701ea0533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:45:21.343608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350860f9-b4de-44c5-adb6-6c7701ea0533 None admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:45:21.343608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350860f9-b4de-44c5-adb6-6c7701ea0533 None admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:45:21.343608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350860f9-b4de-44c5-adb6-6c7701ea0533 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:45:21.343608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350860f9-b4de-44c5-adb6-6c7701ea0533 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:45:21.346826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350860f9-b4de-44c5-adb6-6c7701ea0533 None admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:45:21.459166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 654/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Jul 24 19:45:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:45:31.166607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=30 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:45:31.454273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:06.506052 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34431281-ff12-43b6-a385-b22f3edf9d01 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:06.506869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34431281-ff12-43b6-a385-b22f3edf9d01 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:06.507224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34431281-ff12-43b6-a385-b22f3edf9d01 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:06.515212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a4173-8a96-469a-b16d-7008928d5c20 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:06.515575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a4173-8a96-469a-b16d-7008928d5c20 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:06.515793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a4173-8a96-469a-b16d-7008928d5c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:06.519569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 653/1307] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:06 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:06.611873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-998a4173-8a96-469a-b16d-7008928d5c20 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:06.613042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-998a4173-8a96-469a-b16d-7008928d5c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:06.737140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 655/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:52:06.771233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487200a-3b11-4141-8319-978c8f69ff75 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:06.771524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487200a-3b11-4141-8319-978c8f69ff75 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:06.771798 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487200a-3b11-4141-8319-978c8f69ff75 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:06.773202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 654/1309] 10.222.0.17 () {54 vars in 965 bytes} [Sun Jul 24 19:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:06.779197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b6fd-f6a3-4044-ae09-7bca4b5b70f2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:06.779518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b6fd-f6a3-4044-ae09-7bca4b5b70f2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:06.779808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b6fd-f6a3-4044-ae09-7bca4b5b70f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:06.884638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-1c75b6fd-f6a3-4044-ae09-7bca4b5b70f2 None None] MFA Rules not processed for user `b9763f9899214df3929f3816f009e857`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:06.885801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1c75b6fd-f6a3-4044-ae09-7bca4b5b70f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:06.909396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:06.971408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 656/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Sun Jul 24 19:52:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:06.980816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487200a-3b11-4141-8319-978c8f69ff75 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:06.982868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0487200a-3b11-4141-8319-978c8f69ff75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:07.103041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487200a-3b11-4141-8319-978c8f69ff75 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:07.180043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2b1e4-2aac-4b95-8143-fd2999f76686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:07.181141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2b1e4-2aac-4b95-8143-fd2999f76686 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:07.181630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2b1e4-2aac-4b95-8143-fd2999f76686 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:07.182124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2b1e4-2aac-4b95-8143-fd2999f76686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:07.183979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-dba2b1e4-2aac-4b95-8143-fd2999f76686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:07.242989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:07.301618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2b1e4-2aac-4b95-8143-fd2999f76686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:07.303429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2b1e4-2aac-4b95-8143-fd2999f76686 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:07.323591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 655/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:09.765406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036ebc3-0469-490b-8331-8230d1a86529 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:09.765968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036ebc3-0469-490b-8331-8230d1a86529 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:09.765968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036ebc3-0469-490b-8331-8230d1a86529 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:09.767349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 657/1312] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:09.773222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a750ad-164f-4099-bf04-2e4738b5a585 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:09.773645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a750ad-164f-4099-bf04-2e4738b5a585 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:09.773782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a750ad-164f-4099-bf04-2e4738b5a585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:09.852758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f1a750ad-164f-4099-bf04-2e4738b5a585 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:09.853989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f1a750ad-164f-4099-bf04-2e4738b5a585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:09.949395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 656/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:10.002646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce38dd-c3a5-4b09-8093-f9f5faf5f58b None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:10.003124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce38dd-c3a5-4b09-8093-f9f5faf5f58b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:10.003443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce38dd-c3a5-4b09-8093-f9f5faf5f58b None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:10.005489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 658/1314] 10.222.0.17 () {54 vars in 965 bytes} [Sun Jul 24 19:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:10.011596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612134d-080d-4bd7-8428-06e8ab9caac4 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:10.011896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612134d-080d-4bd7-8428-06e8ab9caac4 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:10.012184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612134d-080d-4bd7-8428-06e8ab9caac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:10.022619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:10.087134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d612134d-080d-4bd7-8428-06e8ab9caac4 None None] MFA Rules not processed for user `b9763f9899214df3929f3816f009e857`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:10.088353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d612134d-080d-4bd7-8428-06e8ab9caac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:10.163865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 657/1315] 10.222.0.17 () {58 vars in 1062 bytes} [Sun Jul 24 19:52:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:52:10.172923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce38dd-c3a5-4b09-8093-f9f5faf5f58b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:10.174535 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-25ce38dd-c3a5-4b09-8093-f9f5faf5f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:10.281764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce38dd-c3a5-4b09-8093-f9f5faf5f58b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:10.359616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:10.362353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ad111-369e-4e74-81de-4b8e9918df8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:10.363017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ad111-369e-4e74-81de-4b8e9918df8d service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:10.363305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ad111-369e-4e74-81de-4b8e9918df8d service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:10.364174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ad111-369e-4e74-81de-4b8e9918df8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:10.365904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-974ad111-369e-4e74-81de-4b8e9918df8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:10.484072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ad111-369e-4e74-81de-4b8e9918df8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:10.485542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ad111-369e-4e74-81de-4b8e9918df8d service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:10.508424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 659/1316] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:13.331963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0e0fd-7eb6-401a-8bd6-2d34f012f73e None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:13.333147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0e0fd-7eb6-401a-8bd6-2d34f012f73e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:13.333147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0e0fd-7eb6-401a-8bd6-2d34f012f73e None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:13.334464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 658/1317] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:13.339548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bc1c2-a8a3-4063-8fca-5722ff4b4da4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:13.339954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bc1c2-a8a3-4063-8fca-5722ff4b4da4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:13.340114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bc1c2-a8a3-4063-8fca-5722ff4b4da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:13.405771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-021bc1c2-a8a3-4063-8fca-5722ff4b4da4 None None] MFA Rules not processed for user `fcdb4126379942f5816db31b2de8f861`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:13.407394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-021bc1c2-a8a3-4063-8fca-5722ff4b4da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:13.461403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:13.503716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 660/1318] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:13.543460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0e0fd-7eb6-401a-8bd6-2d34f012f73e None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:13.545030 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-60b0e0fd-7eb6-401a-8bd6-2d34f012f73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:13.649302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0e0fd-7eb6-401a-8bd6-2d34f012f73e None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:13.652026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:13.723477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a83043-1fd0-4037-9772-30087962aa43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:13.724143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a83043-1fd0-4037-9772-30087962aa43 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:13.724430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a83043-1fd0-4037-9772-30087962aa43 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:13.724737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a83043-1fd0-4037-9772-30087962aa43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:13.726412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d2a83043-1fd0-4037-9772-30087962aa43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:13.834609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a83043-1fd0-4037-9772-30087962aa43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:13.836083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a83043-1fd0-4037-9772-30087962aa43 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:13.853872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 659/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:16.756240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf87cc-23e6-4193-98a4-437742f5d552 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:16.756819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf87cc-23e6-4193-98a4-437742f5d552 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:16.756819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf87cc-23e6-4193-98a4-437742f5d552 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:16.758217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 661/1320] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:16.763620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559535a4-f8ec-4a3e-9efe-bf2b38841dfb None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:16.764000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559535a4-f8ec-4a3e-9efe-bf2b38841dfb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:16.764165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559535a4-f8ec-4a3e-9efe-bf2b38841dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:16.804563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:16.829698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-559535a4-f8ec-4a3e-9efe-bf2b38841dfb None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:16.830902 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-559535a4-f8ec-4a3e-9efe-bf2b38841dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:16.959271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 660/1321] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:16.998282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf87cc-23e6-4193-98a4-437742f5d552 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:16.999795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-68bf87cc-23e6-4193-98a4-437742f5d552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:17.101706 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf87cc-23e6-4193-98a4-437742f5d552 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:17.173192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f811c-ded8-49b4-8bd3-d89aadb16cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:17.173894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f811c-ded8-49b4-8bd3-d89aadb16cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:17.174179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f811c-ded8-49b4-8bd3-d89aadb16cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:17.174451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f811c-ded8-49b4-8bd3-d89aadb16cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:17.176954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-739f811c-ded8-49b4-8bd3-d89aadb16cc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:17.184414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:17.283064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f811c-ded8-49b4-8bd3-d89aadb16cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:17.284523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f811c-ded8-49b4-8bd3-d89aadb16cc5 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:17.303278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 662/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:20.728345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89836eb-5347-49ff-85cf-25800102114d None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:20.728996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89836eb-5347-49ff-85cf-25800102114d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:20.729093 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89836eb-5347-49ff-85cf-25800102114d None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:20.730421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 661/1323] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:20.735776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24342437-7942-4126-9c45-5cd035644909 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:20.736263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24342437-7942-4126-9c45-5cd035644909 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:20.736361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24342437-7942-4126-9c45-5cd035644909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:20.803644 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-24342437-7942-4126-9c45-5cd035644909 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:20.804879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-24342437-7942-4126-9c45-5cd035644909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:20.864913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:20.907087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 663/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:52:20.943816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89836eb-5347-49ff-85cf-25800102114d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:20.945424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a89836eb-5347-49ff-85cf-25800102114d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:21.071283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89836eb-5347-49ff-85cf-25800102114d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:21.084537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:21.151561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb2dc6-66b2-4d85-be47-5dc39cbc4b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:21.152217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb2dc6-66b2-4d85-be47-5dc39cbc4b11 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:21.152506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb2dc6-66b2-4d85-be47-5dc39cbc4b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:21.152821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb2dc6-66b2-4d85-be47-5dc39cbc4b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:21.154483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-58bb2dc6-66b2-4d85-be47-5dc39cbc4b11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:21.264937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb2dc6-66b2-4d85-be47-5dc39cbc4b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:21.266393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb2dc6-66b2-4d85-be47-5dc39cbc4b11 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:21.284624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 662/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:23.452575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ac0ff-020c-46af-94da-5a56e599c3ee None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:23.453211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ac0ff-020c-46af-94da-5a56e599c3ee None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:23.453211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ac0ff-020c-46af-94da-5a56e599c3ee None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:23.454538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 664/1326] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:23.459766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177440c-3054-4599-9eb8-c8582551ef53 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:23.460180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177440c-3054-4599-9eb8-c8582551ef53 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:23.460352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177440c-3054-4599-9eb8-c8582551ef53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:23.506808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:23.525256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-6177440c-3054-4599-9eb8-c8582551ef53 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:23.526520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6177440c-3054-4599-9eb8-c8582551ef53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:23.626032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 663/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:23.663771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ac0ff-020c-46af-94da-5a56e599c3ee None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:23.663771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c23ac0ff-020c-46af-94da-5a56e599c3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:23.774845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ac0ff-020c-46af-94da-5a56e599c3ee None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:23.854136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be267207-5118-4555-8474-59443570bd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:23.854786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267207-5118-4555-8474-59443570bd4f service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:23.855070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267207-5118-4555-8474-59443570bd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:23.855344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267207-5118-4555-8474-59443570bd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:23.857010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-be267207-5118-4555-8474-59443570bd4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:23.874072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:23.967440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be267207-5118-4555-8474-59443570bd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:23.968924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be267207-5118-4555-8474-59443570bd4f service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:23.993840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 665/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:26.095971 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61898b9f-ee69-4c67-942d-286c1dd416ea None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:26.097003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61898b9f-ee69-4c67-942d-286c1dd416ea None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:26.097286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61898b9f-ee69-4c67-942d-286c1dd416ea None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:26.098732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 664/1329] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:26.104432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f99a-65b8-4541-9ee8-55abd6d28f34 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:26.104991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f99a-65b8-4541-9ee8-55abd6d28f34 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:26.105160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f99a-65b8-4541-9ee8-55abd6d28f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:26.171962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-4213f99a-65b8-4541-9ee8-55abd6d28f34 None None] MFA Rules not processed for user `fcdb4126379942f5816db31b2de8f861`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:26.173292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4213f99a-65b8-4541-9ee8-55abd6d28f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:26.245694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:26.277971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 666/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:26.314970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61898b9f-ee69-4c67-942d-286c1dd416ea None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:26.316554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-61898b9f-ee69-4c67-942d-286c1dd416ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:26.421099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61898b9f-ee69-4c67-942d-286c1dd416ea None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:26.443865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:26.496966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887f71d-bc6c-4803-b43a-97f1cbd5b904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:26.497611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887f71d-bc6c-4803-b43a-97f1cbd5b904 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:26.497894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887f71d-bc6c-4803-b43a-97f1cbd5b904 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:26.498190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887f71d-bc6c-4803-b43a-97f1cbd5b904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:26.499789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9887f71d-bc6c-4803-b43a-97f1cbd5b904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:26.626826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887f71d-bc6c-4803-b43a-97f1cbd5b904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:26.628480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887f71d-bc6c-4803-b43a-97f1cbd5b904 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:26.649330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 665/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:26 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:29.206194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d9add-d861-4e72-b981-f90bbc861fb9 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:29.206758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d9add-d861-4e72-b981-f90bbc861fb9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:29.206758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d9add-d861-4e72-b981-f90bbc861fb9 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:29.208722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 667/1332] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:29.214216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae5853-0fd2-484c-bd85-db61a229fa5f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:29.214704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae5853-0fd2-484c-bd85-db61a229fa5f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:29.214795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae5853-0fd2-484c-bd85-db61a229fa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:29.276210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:29.278764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-31ae5853-0fd2-484c-bd85-db61a229fa5f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:29.279967 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-31ae5853-0fd2-484c-bd85-db61a229fa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:29.376703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 666/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:29.412988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d9add-d861-4e72-b981-f90bbc861fb9 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:29.414523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c56d9add-d861-4e72-b981-f90bbc861fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:29.519543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d9add-d861-4e72-b981-f90bbc861fb9 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:29.592326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676b26b-4645-415e-962a-e72c92f686b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:29.592942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676b26b-4645-415e-962a-e72c92f686b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:29.593228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676b26b-4645-415e-962a-e72c92f686b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:29.593543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676b26b-4645-415e-962a-e72c92f686b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:29.595157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c676b26b-4645-415e-962a-e72c92f686b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:29.622469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:29.705284 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676b26b-4645-415e-962a-e72c92f686b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:29.706710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676b26b-4645-415e-962a-e72c92f686b2 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:29.724655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 668/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:31.800816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b38d5-e691-44fd-9c87-8ca56dd65f07 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:31.801399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b38d5-e691-44fd-9c87-8ca56dd65f07 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:31.801399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b38d5-e691-44fd-9c87-8ca56dd65f07 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:31.802868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 667/1335] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:31.808276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dc6ea-62d1-4da7-88d0-38f09367257d None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:31.808739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dc6ea-62d1-4da7-88d0-38f09367257d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:31.808876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dc6ea-62d1-4da7-88d0-38f09367257d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:31.873695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f72dc6ea-62d1-4da7-88d0-38f09367257d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:31.874906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f72dc6ea-62d1-4da7-88d0-38f09367257d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:31.948054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:31.970508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 669/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:32.011152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b38d5-e691-44fd-9c87-8ca56dd65f07 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:32.012699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-017b38d5-e691-44fd-9c87-8ca56dd65f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:32.115529 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b38d5-e691-44fd-9c87-8ca56dd65f07 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:32.143941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:32.188774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02a78f-84b7-42b3-9f65-92fc331cf2ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:32.189418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02a78f-84b7-42b3-9f65-92fc331cf2ab service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:32.189705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02a78f-84b7-42b3-9f65-92fc331cf2ab service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:32.189988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02a78f-84b7-42b3-9f65-92fc331cf2ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:32.192317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fe02a78f-84b7-42b3-9f65-92fc331cf2ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:32.299331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02a78f-84b7-42b3-9f65-92fc331cf2ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:32.301145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02a78f-84b7-42b3-9f65-92fc331cf2ab service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:32.319894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 668/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:35.088867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e392b64-de5c-44cf-b35d-148d7abed655 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:35.089432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e392b64-de5c-44cf-b35d-148d7abed655 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:35.089432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e392b64-de5c-44cf-b35d-148d7abed655 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:35.090788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 670/1338] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:35.095975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d4d18-c41f-49cb-87f2-05a90acb61d4 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:35.096409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d4d18-c41f-49cb-87f2-05a90acb61d4 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:35.096531 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d4d18-c41f-49cb-87f2-05a90acb61d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:35.162188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-796d4d18-c41f-49cb-87f2-05a90acb61d4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:35.163399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-796d4d18-c41f-49cb-87f2-05a90acb61d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:35.171815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:35.260112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 669/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:52:35.296106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e392b64-de5c-44cf-b35d-148d7abed655 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:35.297667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4e392b64-de5c-44cf-b35d-148d7abed655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:35.401148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e392b64-de5c-44cf-b35d-148d7abed655 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:35.479936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d7ee1-76e2-4d0a-958a-c55b1abc878c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:35.480703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d7ee1-76e2-4d0a-958a-c55b1abc878c service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:35.481003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d7ee1-76e2-4d0a-958a-c55b1abc878c service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:35.481280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d7ee1-76e2-4d0a-958a-c55b1abc878c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:35.482941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-633d7ee1-76e2-4d0a-958a-c55b1abc878c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:35.525604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:35.598284 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d7ee1-76e2-4d0a-958a-c55b1abc878c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:35.599713 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d7ee1-76e2-4d0a-958a-c55b1abc878c service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:35.618829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 671/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:40.021881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07855ef-c041-4703-a980-e800ca0bb59a None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:40.022397 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07855ef-c041-4703-a980-e800ca0bb59a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:40.022488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07855ef-c041-4703-a980-e800ca0bb59a None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:40.023975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 670/1341] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:40.029320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2655b-a5ac-4ec5-9023-c9eab56e5816 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:40.029729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2655b-a5ac-4ec5-9023-c9eab56e5816 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:40.029890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2655b-a5ac-4ec5-9023-c9eab56e5816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:40.109340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-09a2655b-a5ac-4ec5-9023-c9eab56e5816 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:40.110563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-09a2655b-a5ac-4ec5-9023-c9eab56e5816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:40.206670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:40.224694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 672/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:40.261081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07855ef-c041-4703-a980-e800ca0bb59a None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:40.262615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e07855ef-c041-4703-a980-e800ca0bb59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:40.365924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07855ef-c041-4703-a980-e800ca0bb59a None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:40.403441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:40.446713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f810b6d-90b6-4069-9b50-fb9a05b4962e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:40.447402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f810b6d-90b6-4069-9b50-fb9a05b4962e service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:40.447693 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f810b6d-90b6-4069-9b50-fb9a05b4962e service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:40.448048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f810b6d-90b6-4069-9b50-fb9a05b4962e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:40.449781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9f810b6d-90b6-4069-9b50-fb9a05b4962e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:40.569526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f810b6d-90b6-4069-9b50-fb9a05b4962e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:40.571220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f810b6d-90b6-4069-9b50-fb9a05b4962e service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:40.590277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 671/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:43.642135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ff3c-8d6b-4983-b84f-2ae965197e37 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:43.642666 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ff3c-8d6b-4983-b84f-2ae965197e37 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:43.642757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ff3c-8d6b-4983-b84f-2ae965197e37 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:43.644095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 673/1344] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:43.649931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a09d44-4258-446a-a2e4-1311487b85c5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:43.650443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a09d44-4258-446a-a2e4-1311487b85c5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:43.650549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a09d44-4258-446a-a2e4-1311487b85c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:43.716758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-56a09d44-4258-446a-a2e4-1311487b85c5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:43.717883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-56a09d44-4258-446a-a2e4-1311487b85c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:43.741770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:43.822954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 672/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:43.864941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9ff3c-8d6b-4983-b84f-2ae965197e37 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:43.866469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6cf9ff3c-8d6b-4983-b84f-2ae965197e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:43.974737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9ff3c-8d6b-4983-b84f-2ae965197e37 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:44.050042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b348a4d-7bd5-4626-a23a-f144cfc24afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:44.050956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b348a4d-7bd5-4626-a23a-f144cfc24afe service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:44.051236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b348a4d-7bd5-4626-a23a-f144cfc24afe service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:44.051508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b348a4d-7bd5-4626-a23a-f144cfc24afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:44.053156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4b348a4d-7bd5-4626-a23a-f144cfc24afe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:44.098494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:44.167471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b348a4d-7bd5-4626-a23a-f144cfc24afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:44.168974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b348a4d-7bd5-4626-a23a-f144cfc24afe service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:44.189758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 674/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:48.672774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f29eef-2d51-45c2-91cc-915625ec8052 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:48.673242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f29eef-2d51-45c2-91cc-915625ec8052 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:48.673366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f29eef-2d51-45c2-91cc-915625ec8052 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:48.674938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 673/1347] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:48.681219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76324977-aab7-4532-ad0d-89f4b519ac4d None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:48.681732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76324977-aab7-4532-ad0d-89f4b519ac4d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:48.681832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76324977-aab7-4532-ad0d-89f4b519ac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:48.768599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-76324977-aab7-4532-ad0d-89f4b519ac4d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:48.770012 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-76324977-aab7-4532-ad0d-89f4b519ac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:48.880478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:48.883117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 675/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:48.929258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f29eef-2d51-45c2-91cc-915625ec8052 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:52:48.930796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-69f29eef-2d51-45c2-91cc-915625ec8052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:49.046680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f29eef-2d51-45c2-91cc-915625ec8052 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:52:49.099932 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:49.126941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad519e0-e823-4cfc-9a2d-f0111fcbedc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:52:49.127600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad519e0-e823-4cfc-9a2d-f0111fcbedc9 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:49.128032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad519e0-e823-4cfc-9a2d-f0111fcbedc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:49.128514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad519e0-e823-4cfc-9a2d-f0111fcbedc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:49.130729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0ad519e0-e823-4cfc-9a2d-f0111fcbedc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:49.243064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad519e0-e823-4cfc-9a2d-f0111fcbedc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:52:49.244505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad519e0-e823-4cfc-9a2d-f0111fcbedc9 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:52:49.264539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 674/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:52:54.213398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b5509-1cfe-4c4f-b1cb-99996b5bb1fc None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:54.213900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b5509-1cfe-4c4f-b1cb-99996b5bb1fc None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:54.214001 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b5509-1cfe-4c4f-b1cb-99996b5bb1fc None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:54.215525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 676/1350] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 19:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:54.220591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42371113-897d-4248-9465-949523054699 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:54.221063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42371113-897d-4248-9465-949523054699 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:54.221172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42371113-897d-4248-9465-949523054699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:54.338064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-42371113-897d-4248-9465-949523054699 None None] MFA Rules not processed for user `1c1b23b83544423abc6919d805916b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:54.339201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-42371113-897d-4248-9465-949523054699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:54.373096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:54.499699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 675/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 19:52:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:52:58.694746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a502b-974a-4913-8238-925651c3c4d3 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:58.695235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a502b-974a-4913-8238-925651c3c4d3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:58.695235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a502b-974a-4913-8238-925651c3c4d3 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:58.696631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 677/1352] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 19:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:52:58.702066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f10be-40e9-4cee-8c45-644e4620fd86 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:52:58.702467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f10be-40e9-4cee-8c45-644e4620fd86 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:52:58.702631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f10be-40e9-4cee-8c45-644e4620fd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:52:58.784410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-547f10be-40e9-4cee-8c45-644e4620fd86 None None] MFA Rules not processed for user `1c1b23b83544423abc6919d805916b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:52:58.785934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-547f10be-40e9-4cee-8c45-644e4620fd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:52:58.864809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:52:58.903427 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 676/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 19:52:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:00.261346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b4796-f74c-4898-a1a1-54ea87b330c2 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:00.261875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b4796-f74c-4898-a1a1-54ea87b330c2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:00.261875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b4796-f74c-4898-a1a1-54ea87b330c2 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:00.263266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 678/1354] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 19:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:00.268375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72103cb7-53b2-47be-a61f-74720ca9ba23 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:00.269187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72103cb7-53b2-47be-a61f-74720ca9ba23 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:00.269505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72103cb7-53b2-47be-a61f-74720ca9ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:00.345342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-72103cb7-53b2-47be-a61f-74720ca9ba23 None None] MFA Rules not processed for user `1c1b23b83544423abc6919d805916b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:00.346586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-72103cb7-53b2-47be-a61f-74720ca9ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:00.459600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 677/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 19:53:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:04.025729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c40c7e-aa59-4dc3-be93-f01952b217e5 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:04.026291 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c40c7e-aa59-4dc3-be93-f01952b217e5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:04.026291 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c40c7e-aa59-4dc3-be93-f01952b217e5 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:04.027794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 679/1356] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:04.033257 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed069a-c487-4302-8fbb-d6f8931dedda None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:04.033722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed069a-c487-4302-8fbb-d6f8931dedda None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:04.033894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed069a-c487-4302-8fbb-d6f8931dedda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:04.070867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:04.103085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-49ed069a-c487-4302-8fbb-d6f8931dedda None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:04.104207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-49ed069a-c487-4302-8fbb-d6f8931dedda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:04.206889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 678/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:04.264339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6391e-d07f-4a28-8f30-7fdf41008bfe None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:04.265234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6391e-d07f-4a28-8f30-7fdf41008bfe None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:04.265735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6391e-d07f-4a28-8f30-7fdf41008bfe None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:04.267901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 680/1358] 10.222.0.17 () {54 vars in 962 bytes} [Sun Jul 24 19:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:04.275700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c833296-42d7-48c1-aa99-b1f1d057d205 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:04.276320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c833296-42d7-48c1-aa99-b1f1d057d205 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:04.277166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c833296-42d7-48c1-aa99-b1f1d057d205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:04.363190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1c833296-42d7-48c1-aa99-b1f1d057d205 None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:04.365396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1c833296-42d7-48c1-aa99-b1f1d057d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:04.420875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:04.507124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 679/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Jul 24 19:53:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:04.515655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6391e-d07f-4a28-8f30-7fdf41008bfe None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:04.517236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b6d6391e-d07f-4a28-8f30-7fdf41008bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:04.681518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6391e-d07f-4a28-8f30-7fdf41008bfe None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:04.768539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:04.812288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e55cc-d664-45d9-8d5e-2cf28aba9b7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:04.812976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e55cc-d664-45d9-8d5e-2cf28aba9b7a service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:04.813261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e55cc-d664-45d9-8d5e-2cf28aba9b7a service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:04.813540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e55cc-d664-45d9-8d5e-2cf28aba9b7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:04.815206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fc8e55cc-d664-45d9-8d5e-2cf28aba9b7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:04.929031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e55cc-d664-45d9-8d5e-2cf28aba9b7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:04.930611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e55cc-d664-45d9-8d5e-2cf28aba9b7a service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:04.949245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 681/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:07.252483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9da9-b031-4ad3-af22-f909737f9677 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:07.253021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9da9-b031-4ad3-af22-f909737f9677 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:07.253021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9da9-b031-4ad3-af22-f909737f9677 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:07.255193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 680/1361] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:07.260555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbb2b3-430f-4f0a-be2a-2ed7a1fffa90 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:07.261125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbb2b3-430f-4f0a-be2a-2ed7a1fffa90 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:07.261213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbb2b3-430f-4f0a-be2a-2ed7a1fffa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:07.331036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-57cbb2b3-430f-4f0a-be2a-2ed7a1fffa90 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:07.332203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-57cbb2b3-430f-4f0a-be2a-2ed7a1fffa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:07.376597 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:07.445370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 682/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:53:07.490470 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c9da9-b031-4ad3-af22-f909737f9677 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:07.492226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5e9c9da9-b031-4ad3-af22-f909737f9677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:07.702470 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c9da9-b031-4ad3-af22-f909737f9677 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:07.712699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:07.827540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7d3e0-a027-41c7-8e1a-ea41c16fb778 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:07.828213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7d3e0-a027-41c7-8e1a-ea41c16fb778 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:07.828505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7d3e0-a027-41c7-8e1a-ea41c16fb778 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:07.828821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7d3e0-a027-41c7-8e1a-ea41c16fb778 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:07.830513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3ae7d3e0-a027-41c7-8e1a-ea41c16fb778 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:07.936612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7d3e0-a027-41c7-8e1a-ea41c16fb778 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:07.938914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7d3e0-a027-41c7-8e1a-ea41c16fb778 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:07.964155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 681/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:10.573651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2313d-9baa-471e-b829-6d58a962a182 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:10.574548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2313d-9baa-471e-b829-6d58a962a182 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:10.574827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2313d-9baa-471e-b829-6d58a962a182 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:10.576220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 683/1364] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:53:10.582076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd649f-6fac-45d6-90e2-823c0c9c9306 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:10.583077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd649f-6fac-45d6-90e2-823c0c9c9306 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:10.583182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd649f-6fac-45d6-90e2-823c0c9c9306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:10.599152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:10.654407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a1dd649f-6fac-45d6-90e2-823c0c9c9306 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:10.655679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a1dd649f-6fac-45d6-90e2-823c0c9c9306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:10.773683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 682/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:10.822372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e7301-4acc-4bb6-b9b5-4a1330c7bce1 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:10.822664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e7301-4acc-4bb6-b9b5-4a1330c7bce1 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:10.823131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e7301-4acc-4bb6-b9b5-4a1330c7bce1 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:10.824628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 684/1366] 10.222.0.17 () {54 vars in 962 bytes} [Sun Jul 24 19:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:10.830897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2cc2d-9bd0-44a3-9637-d7a706ca607c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:10.831363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2cc2d-9bd0-44a3-9637-d7a706ca607c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:10.831908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2cc2d-9bd0-44a3-9637-d7a706ca607c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:10.920786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-aad2cc2d-9bd0-44a3-9637-d7a706ca607c None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:10.922195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-aad2cc2d-9bd0-44a3-9637-d7a706ca607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:10.965740 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:11.060319 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 683/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Jul 24 19:53:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:11.068510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e7301-4acc-4bb6-b9b5-4a1330c7bce1 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:11.070136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-685e7301-4acc-4bb6-b9b5-4a1330c7bce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:11.252762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e7301-4acc-4bb6-b9b5-4a1330c7bce1 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:11.278667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:11.397361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f18b61-0892-4d4c-97d4-b9cc102cc5a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:11.398128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f18b61-0892-4d4c-97d4-b9cc102cc5a1 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:11.398546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f18b61-0892-4d4c-97d4-b9cc102cc5a1 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:11.398957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f18b61-0892-4d4c-97d4-b9cc102cc5a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:11.401439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f5f18b61-0892-4d4c-97d4-b9cc102cc5a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:11.532360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f18b61-0892-4d4c-97d4-b9cc102cc5a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:11.535211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f18b61-0892-4d4c-97d4-b9cc102cc5a1 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:11.560804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 685/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:12.416301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7344-5d15-4398-b22e-dcea4ced926b None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:12.416929 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7344-5d15-4398-b22e-dcea4ced926b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:12.417376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7344-5d15-4398-b22e-dcea4ced926b None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:12.420724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 684/1369] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:12.432331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb4a69-7450-4231-8875-3daa38a0f0b7 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:12.433003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb4a69-7450-4231-8875-3daa38a0f0b7 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:12.433608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb4a69-7450-4231-8875-3daa38a0f0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:12.482202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:12.527433 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-42eb4a69-7450-4231-8875-3daa38a0f0b7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:12.528921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-42eb4a69-7450-4231-8875-3daa38a0f0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:12.658641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 686/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:13.099518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd3fd8-664a-49a9-8c07-eef74f0eccef None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:13.099878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd3fd8-664a-49a9-8c07-eef74f0eccef None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:13.100070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd3fd8-664a-49a9-8c07-eef74f0eccef None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:13.101505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 685/1371] 10.222.0.17 () {54 vars in 964 bytes} [Sun Jul 24 19:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:13.106971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e81ef-82b1-443f-a76b-bdc9b5dde203 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:13.107404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e81ef-82b1-443f-a76b-bdc9b5dde203 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:13.107893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e81ef-82b1-443f-a76b-bdc9b5dde203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:13.200785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-169e81ef-82b1-443f-a76b-bdc9b5dde203 None None] MFA Rules not processed for user `fe7a67bd815147afa3f186e95b004469`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:13.202412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-169e81ef-82b1-443f-a76b-bdc9b5dde203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:13.266250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:13.298261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 687/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Sun Jul 24 19:53:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:13.306710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd3fd8-664a-49a9-8c07-eef74f0eccef None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:13.308415 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-22cd3fd8-664a-49a9-8c07-eef74f0eccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:13.440565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd3fd8-664a-49a9-8c07-eef74f0eccef None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:13.490257 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:13.528933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e2312-7f89-4ed5-82fd-ff33d14ae17e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:13.529616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e2312-7f89-4ed5-82fd-ff33d14ae17e service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:13.529907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e2312-7f89-4ed5-82fd-ff33d14ae17e service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:13.530185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e2312-7f89-4ed5-82fd-ff33d14ae17e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:13.531939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-671e2312-7f89-4ed5-82fd-ff33d14ae17e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:13.665589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e2312-7f89-4ed5-82fd-ff33d14ae17e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:13.670079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e2312-7f89-4ed5-82fd-ff33d14ae17e service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:13.692771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 686/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:14.335790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98c336-02c0-446c-9b06-f5181d79ef60 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:14.336409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98c336-02c0-446c-9b06-f5181d79ef60 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:14.336525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98c336-02c0-446c-9b06-f5181d79ef60 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:14.337997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 688/1374] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:14.343582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65758586-ed88-4dcd-9e0d-01e3bc75b8a7 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:14.343879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65758586-ed88-4dcd-9e0d-01e3bc75b8a7 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:14.344162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65758586-ed88-4dcd-9e0d-01e3bc75b8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:14.428305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-65758586-ed88-4dcd-9e0d-01e3bc75b8a7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:14.430134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-65758586-ed88-4dcd-9e0d-01e3bc75b8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:14.468298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:14.583443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 687/1375] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:14.629195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98c336-02c0-446c-9b06-f5181d79ef60 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:14.630786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0d98c336-02c0-446c-9b06-f5181d79ef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:14.781125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98c336-02c0-446c-9b06-f5181d79ef60 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:14.835401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:14.898683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a982a5-c6d6-47b3-a64f-8131f00577b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:14.899256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a982a5-c6d6-47b3-a64f-8131f00577b6 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:14.899539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a982a5-c6d6-47b3-a64f-8131f00577b6 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:14.899813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a982a5-c6d6-47b3-a64f-8131f00577b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:14.901514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d5a982a5-c6d6-47b3-a64f-8131f00577b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:14.990384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a982a5-c6d6-47b3-a64f-8131f00577b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:14.991982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a982a5-c6d6-47b3-a64f-8131f00577b6 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:15.010814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 689/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:17.256119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301affb-9cb4-4dda-847a-bc05189b7e29 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:17.256686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301affb-9cb4-4dda-847a-bc05189b7e29 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:17.256780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301affb-9cb4-4dda-847a-bc05189b7e29 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:17.258163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 688/1377] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:17.263645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f08ed-4221-4e0d-8902-9af267678983 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:17.264059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f08ed-4221-4e0d-8902-9af267678983 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:17.264320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f08ed-4221-4e0d-8902-9af267678983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:17.329141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b49f08ed-4221-4e0d-8902-9af267678983 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:17.330343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b49f08ed-4221-4e0d-8902-9af267678983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:17.337456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:17.447285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 690/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:17.500343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301affb-9cb4-4dda-847a-bc05189b7e29 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:17.501900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8301affb-9cb4-4dda-847a-bc05189b7e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:17.706433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301affb-9cb4-4dda-847a-bc05189b7e29 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:17.710681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:17.831077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bacf5-652e-48d7-93d3-2a3e602c5595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:17.831650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bacf5-652e-48d7-93d3-2a3e602c5595 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:17.831935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bacf5-652e-48d7-93d3-2a3e602c5595 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:17.832803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bacf5-652e-48d7-93d3-2a3e602c5595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:17.834432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d13bacf5-652e-48d7-93d3-2a3e602c5595 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:17.940248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bacf5-652e-48d7-93d3-2a3e602c5595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:17.942421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bacf5-652e-48d7-93d3-2a3e602c5595 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:17.962460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 689/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:20.231866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eee0a-6f25-492e-bbfb-339112eeceae None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:20.231866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eee0a-6f25-492e-bbfb-339112eeceae None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:20.231866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eee0a-6f25-492e-bbfb-339112eeceae None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:20.231866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 691/1380] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:53:20.232779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0cd40-6673-448d-a04c-81b9df2ddaae None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:20.232779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0cd40-6673-448d-a04c-81b9df2ddaae None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:20.232779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0cd40-6673-448d-a04c-81b9df2ddaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:20.234636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:20.303762 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-05a0cd40-6673-448d-a04c-81b9df2ddaae None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:20.305040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-05a0cd40-6673-448d-a04c-81b9df2ddaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:20.412789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 690/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:20.452824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6eee0a-6f25-492e-bbfb-339112eeceae None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:20.454369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2d6eee0a-6f25-492e-bbfb-339112eeceae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:20.614323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:20.625172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6eee0a-6f25-492e-bbfb-339112eeceae None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:20.761435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4fd67-0ad7-441a-a1e1-dd517bcc4b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:20.762247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4fd67-0ad7-441a-a1e1-dd517bcc4b9a service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:20.762507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4fd67-0ad7-441a-a1e1-dd517bcc4b9a service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:20.762790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4fd67-0ad7-441a-a1e1-dd517bcc4b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:20.764469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-89b4fd67-0ad7-441a-a1e1-dd517bcc4b9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:20.868593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4fd67-0ad7-441a-a1e1-dd517bcc4b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:20.870176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4fd67-0ad7-441a-a1e1-dd517bcc4b9a service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:20.881354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:20.890007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 692/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:23.490139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa417999-e9eb-4337-9ab3-fc3bdccbf778 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:23.490689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa417999-e9eb-4337-9ab3-fc3bdccbf778 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:23.490859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa417999-e9eb-4337-9ab3-fc3bdccbf778 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:23.493146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 691/1383] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:23.501481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eae9d-2c5e-4432-a998-d4f0cd3a1b49 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:23.502022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eae9d-2c5e-4432-a998-d4f0cd3a1b49 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:23.502276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eae9d-2c5e-4432-a998-d4f0cd3a1b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:23.576585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-7f8eae9d-2c5e-4432-a998-d4f0cd3a1b49 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:23.577889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7f8eae9d-2c5e-4432-a998-d4f0cd3a1b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:23.690399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 693/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:23.740957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa417999-e9eb-4337-9ab3-fc3bdccbf778 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:23.742482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-aa417999-e9eb-4337-9ab3-fc3bdccbf778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:23.855787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:23.925692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa417999-e9eb-4337-9ab3-fc3bdccbf778 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:24.043559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672be08-97d0-4988-a3a9-43885d02b938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:24.044265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672be08-97d0-4988-a3a9-43885d02b938 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:24.044536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672be08-97d0-4988-a3a9-43885d02b938 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:24.044853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672be08-97d0-4988-a3a9-43885d02b938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:24.046570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3672be08-97d0-4988-a3a9-43885d02b938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:24.101860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:24.147856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672be08-97d0-4988-a3a9-43885d02b938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:24.150403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672be08-97d0-4988-a3a9-43885d02b938 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:24.172525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 692/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:26.599000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da61c3d-4b81-4b9a-bdb2-62c053b0f3bc None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:26.599882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da61c3d-4b81-4b9a-bdb2-62c053b0f3bc None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:26.599882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da61c3d-4b81-4b9a-bdb2-62c053b0f3bc None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:26.601652 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 694/1386] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:53:26.607803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84488800-c6d0-4e8a-8fa1-54452fb6017b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:26.608298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84488800-c6d0-4e8a-8fa1-54452fb6017b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:26.608393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84488800-c6d0-4e8a-8fa1-54452fb6017b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:26.682894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-84488800-c6d0-4e8a-8fa1-54452fb6017b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:26.685040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-84488800-c6d0-4e8a-8fa1-54452fb6017b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:26.805443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 693/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:26.851759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da61c3d-4b81-4b9a-bdb2-62c053b0f3bc None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:26.853320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5da61c3d-4b81-4b9a-bdb2-62c053b0f3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:26.933832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:27.032471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da61c3d-4b81-4b9a-bdb2-62c053b0f3bc None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:27.151408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15b312-f4a5-432a-a890-f09b7ae36892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:27.152710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15b312-f4a5-432a-a890-f09b7ae36892 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:27.152998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15b312-f4a5-432a-a890-f09b7ae36892 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:27.153275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15b312-f4a5-432a-a890-f09b7ae36892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:27.154928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9d15b312-f4a5-432a-a890-f09b7ae36892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:27.198423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:27.256894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15b312-f4a5-432a-a890-f09b7ae36892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:27.258504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15b312-f4a5-432a-a890-f09b7ae36892 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:27.278877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 695/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:29.450675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df61c98-073f-43f5-b22c-e1d1d7885d91 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:29.451210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df61c98-073f-43f5-b22c-e1d1d7885d91 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:29.451300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df61c98-073f-43f5-b22c-e1d1d7885d91 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:29.452634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 694/1389] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:29.458545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f700043-415d-40e0-aaac-8f2d07f1ba8e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:29.459021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f700043-415d-40e0-aaac-8f2d07f1ba8e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:29.459120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f700043-415d-40e0-aaac-8f2d07f1ba8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:29.524939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e2b1f-a9a3-4dc7-b4f0-c9a2c02c193c None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:29.525246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e2b1f-a9a3-4dc7-b4f0-c9a2c02c193c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:29.525493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e2b1f-a9a3-4dc7-b4f0-c9a2c02c193c None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:29.525703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8f700043-415d-40e0-aaac-8f2d07f1ba8e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:29.526850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 695/1390] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:53:29.527037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8f700043-415d-40e0-aaac-8f2d07f1ba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:29.531922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fee30f-6f4f-4265-8d50-b18a5dd793d3 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:29.532212 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fee30f-6f4f-4265-8d50-b18a5dd793d3 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:29.532492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fee30f-6f4f-4265-8d50-b18a5dd793d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:29.582236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:29.617345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-78fee30f-6f4f-4265-8d50-b18a5dd793d3 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:29.618627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-78fee30f-6f4f-4265-8d50-b18a5dd793d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:29.632967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 696/1391] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:29.689872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cf015-8155-4057-a094-db1fe70ae271 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:29.690166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cf015-8155-4057-a094-db1fe70ae271 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:29.690456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cf015-8155-4057-a094-db1fe70ae271 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:29.691879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 697/1392] 10.222.0.17 () {54 vars in 966 bytes} [Sun Jul 24 19:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:53:29.696553 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ab4a-9e9a-4d2b-85b7-bce498e70ce5 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:29.696869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ab4a-9e9a-4d2b-85b7-bce498e70ce5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:29.697154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ab4a-9e9a-4d2b-85b7-bce498e70ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:29.711987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:29.720638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 696/1393] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:53:29.776142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fee30f-6f4f-4265-8d50-b18a5dd793d3 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:29.777874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-78fee30f-6f4f-4265-8d50-b18a5dd793d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:29.780022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-9d68ab4a-9e9a-4d2b-85b7-bce498e70ce5 None None] MFA Rules not processed for user `bca33ef49ba44832a7fb45b2317d211c`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:29.781225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9d68ab4a-9e9a-4d2b-85b7-bce498e70ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:29.863972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 698/1394] 10.222.0.17 () {58 vars in 1063 bytes} [Sun Jul 24 19:53:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:53:29.872122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68ab4a-9e9a-4d2b-85b7-bce498e70ce5 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:29.880367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68ab4a-9e9a-4d2b-85b7-bce498e70ce5 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:29.886394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:29.893868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fee30f-6f4f-4265-8d50-b18a5dd793d3 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:29.957378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68713e-a40c-40ae-9859-1ff0afd4a682 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca33ef49ba44832a7fb45b2317d211c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:29.957957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68713e-a40c-40ae-9859-1ff0afd4a682 service glance] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:29.958243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68713e-a40c-40ae-9859-1ff0afd4a682 service glance] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:29.958518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68713e-a40c-40ae-9859-1ff0afd4a682 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:29.960168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ca68713e-a40c-40ae-9859-1ff0afd4a682 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:29.991139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:30.017058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5201156-8691-4ce8-aafd-417117442fc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:30.017725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5201156-8691-4ce8-aafd-417117442fc5 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:30.018011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5201156-8691-4ce8-aafd-417117442fc5 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:30.018315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5201156-8691-4ce8-aafd-417117442fc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:30.020705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c5201156-8691-4ce8-aafd-417117442fc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:30.076970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68713e-a40c-40ae-9859-1ff0afd4a682 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:30.078424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68713e-a40c-40ae-9859-1ff0afd4a682 service glance] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:30.097940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 699/1395] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 19:53:30.111732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5201156-8691-4ce8-aafd-417117442fc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:30.113861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5201156-8691-4ce8-aafd-417117442fc5 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:30.129629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:30.132532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 697/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 19:53:31.052498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc6a83-841f-4c07-b0ec-38cfdbfd84e3 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:31.053017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc6a83-841f-4c07-b0ec-38cfdbfd84e3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:31.053130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc6a83-841f-4c07-b0ec-38cfdbfd84e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:31.119281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-18bc6a83-841f-4c07-b0ec-38cfdbfd84e3 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:31.120524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-18bc6a83-841f-4c07-b0ec-38cfdbfd84e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:31.202984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:31.209030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 700/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 19:53:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:31.252609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ab286-2cd2-400d-b861-5c85544f445c None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:31.253580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ab286-2cd2-400d-b861-5c85544f445c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:31.253867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ab286-2cd2-400d-b861-5c85544f445c None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:31.255269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 698/1398] 10.222.0.17 () {54 vars in 943 bytes} [Sun Jul 24 19:53:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:31.261526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277cb963-a9e5-4310-a975-2c3975df2cc4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:31.261823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277cb963-a9e5-4310-a975-2c3975df2cc4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:31.262106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277cb963-a9e5-4310-a975-2c3975df2cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:31.338342 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-277cb963-a9e5-4310-a975-2c3975df2cc4 None None] MFA Rules not processed for user `0849f31902da4613babb10a76c4548f3`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:31.339536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-277cb963-a9e5-4310-a975-2c3975df2cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:31.454030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 701/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Sun Jul 24 19:53:31 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:31.464064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ab286-2cd2-400d-b861-5c85544f445c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:31.465641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-323ab286-2cd2-400d-b861-5c85544f445c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:31.609231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ab286-2cd2-400d-b861-5c85544f445c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:31.693786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:31.713988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803a591-d655-46c6-9e6e-ededf353e257 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:31.714666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803a591-d655-46c6-9e6e-ededf353e257 service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:31.714963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803a591-d655-46c6-9e6e-ededf353e257 service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:31.715244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803a591-d655-46c6-9e6e-ededf353e257 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:31.716972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2803a591-d655-46c6-9e6e-ededf353e257 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:31.807234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803a591-d655-46c6-9e6e-ededf353e257 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:31.808939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803a591-d655-46c6-9e6e-ededf353e257 service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:31.817343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 699/1400] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 19:53:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 19:53:32.415996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a700-9f15-428e-b152-4aeb777a634c None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:32.416507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a700-9f15-428e-b152-4aeb777a634c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:32.416507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a700-9f15-428e-b152-4aeb777a634c None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:32.417962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 702/1401] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:32.423431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0deeb5-38d2-4c0e-865d-1d2eae57c7ee None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:32.423896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0deeb5-38d2-4c0e-865d-1d2eae57c7ee None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:32.423988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0deeb5-38d2-4c0e-865d-1d2eae57c7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:32.494284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2d0deeb5-38d2-4c0e-865d-1d2eae57c7ee None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:32.495671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2d0deeb5-38d2-4c0e-865d-1d2eae57c7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:32.574888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:32.606178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 700/1402] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:32.656496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a700-9f15-428e-b152-4aeb777a634c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:32.658051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-05e4a700-9f15-428e-b152-4aeb777a634c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:32.726783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:32.845057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a700-9f15-428e-b152-4aeb777a634c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:32.990091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2dbf1e-3636-4c8b-b78a-1f6bdd7afab1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:32.990699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2dbf1e-3636-4c8b-b78a-1f6bdd7afab1 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:32.991060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2dbf1e-3636-4c8b-b78a-1f6bdd7afab1 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:32.991344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2dbf1e-3636-4c8b-b78a-1f6bdd7afab1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:32.993036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-de2dbf1e-3636-4c8b-b78a-1f6bdd7afab1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:33.014305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:33.098436 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2dbf1e-3636-4c8b-b78a-1f6bdd7afab1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:33.100086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2dbf1e-3636-4c8b-b78a-1f6bdd7afab1 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:33.120794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 703/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:35.843844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928fd91-7ff2-45ec-8c67-6fa82ef3e460 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:35.844448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928fd91-7ff2-45ec-8c67-6fa82ef3e460 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:35.844448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928fd91-7ff2-45ec-8c67-6fa82ef3e460 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:35.850388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 701/1404] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:35.852683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc4a08-3117-49d5-a2f7-4a85efa60622 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:35.863335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc4a08-3117-49d5-a2f7-4a85efa60622 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:35.863335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc4a08-3117-49d5-a2f7-4a85efa60622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:35.939621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a2cc4a08-3117-49d5-a2f7-4a85efa60622 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:35.941027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a2cc4a08-3117-49d5-a2f7-4a85efa60622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:36.060132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:36.068709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 704/1405] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:53:36.117567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928fd91-7ff2-45ec-8c67-6fa82ef3e460 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:36.119097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6928fd91-7ff2-45ec-8c67-6fa82ef3e460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:36.291354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928fd91-7ff2-45ec-8c67-6fa82ef3e460 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:36.364151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:36.432505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51bde0-eb69-4c17-81e3-c8c97736e876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:36.433299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51bde0-eb69-4c17-81e3-c8c97736e876 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:36.433614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51bde0-eb69-4c17-81e3-c8c97736e876 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:36.433954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51bde0-eb69-4c17-81e3-c8c97736e876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:36.435663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ef51bde0-eb69-4c17-81e3-c8c97736e876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:36.537715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51bde0-eb69-4c17-81e3-c8c97736e876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:36.539456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51bde0-eb69-4c17-81e3-c8c97736e876 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:36.561859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 702/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:38.943281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37018e-e0ab-4189-8297-33df9bb27b0b None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:38.943895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37018e-e0ab-4189-8297-33df9bb27b0b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:38.943895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37018e-e0ab-4189-8297-33df9bb27b0b None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:38.945585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 705/1407] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:38.951028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262abf03-0fbd-4c1d-95e6-17520f929f64 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:38.951475 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262abf03-0fbd-4c1d-95e6-17520f929f64 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:38.951605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262abf03-0fbd-4c1d-95e6-17520f929f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:39.029499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-262abf03-0fbd-4c1d-95e6-17520f929f64 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:39.030662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-262abf03-0fbd-4c1d-95e6-17520f929f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:39.047361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:39.152964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 703/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:39.191847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37018e-e0ab-4189-8297-33df9bb27b0b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:39.193615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ab37018e-e0ab-4189-8297-33df9bb27b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:39.391035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37018e-e0ab-4189-8297-33df9bb27b0b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:39.515984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:39.571300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb7b4-7368-4d5d-9b4a-bc7cc9a70b83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:39.572939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb7b4-7368-4d5d-9b4a-bc7cc9a70b83 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:39.573415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb7b4-7368-4d5d-9b4a-bc7cc9a70b83 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:39.573712 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb7b4-7368-4d5d-9b4a-bc7cc9a70b83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:39.575540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e4edb7b4-7368-4d5d-9b4a-bc7cc9a70b83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:39.676911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edb7b4-7368-4d5d-9b4a-bc7cc9a70b83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:39.678766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edb7b4-7368-4d5d-9b4a-bc7cc9a70b83 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:39.700982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 706/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:53:49.453751 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=37 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:49.695589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=65 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:56.039289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96d35-76ff-48bb-b235-181b91735748 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:56.039878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96d35-76ff-48bb-b235-181b91735748 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:56.039878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96d35-76ff-48bb-b235-181b91735748 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:56.041304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 704/1410] 10.222.0.17 () {54 vars in 918 bytes} [Sun Jul 24 19:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:56.047105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e584-eeec-44b0-83a0-87aaa9ee3b62 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:56.047550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e584-eeec-44b0-83a0-87aaa9ee3b62 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:56.047647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e584-eeec-44b0-83a0-87aaa9ee3b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:56.126992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3328e584-eeec-44b0-83a0-87aaa9ee3b62 None None] MFA Rules not processed for user `1c1b23b83544423abc6919d805916b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:56.128209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3328e584-eeec-44b0-83a0-87aaa9ee3b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:56.237427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 707/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Sun Jul 24 19:53:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:56.251599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305942f-a0d6-44e9-bc65-b832b02a4e0f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:56.251897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305942f-a0d6-44e9-bc65-b832b02a4e0f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:56.252184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305942f-a0d6-44e9-bc65-b832b02a4e0f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:56.253581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 705/1412] 10.222.0.17 () {54 vars in 918 bytes} [Sun Jul 24 19:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:56.258655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bce93-5638-4f40-972a-eb3c22b9acda None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:56.258888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bce93-5638-4f40-972a-eb3c22b9acda None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:56.259165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bce93-5638-4f40-972a-eb3c22b9acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:56.332406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:56.335453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e35bce93-5638-4f40-972a-eb3c22b9acda None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:56.336661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e35bce93-5638-4f40-972a-eb3c22b9acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:56.452217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 708/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Sun Jul 24 19:53:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:58.996732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7219d0d-a67e-4801-b4bb-7bb95a26291f None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:58.997288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7219d0d-a67e-4801-b4bb-7bb95a26291f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:58.997288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7219d0d-a67e-4801-b4bb-7bb95a26291f None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:58.998698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:53:59.004367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec66d90-bf35-442a-af1f-c96c9ca01c25 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:59.004827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec66d90-bf35-442a-af1f-c96c9ca01c25 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:59.004924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec66d90-bf35-442a-af1f-c96c9ca01c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:59.070008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3ec66d90-bf35-442a-af1f-c96c9ca01c25 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:53:59.071205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3ec66d90-bf35-442a-af1f-c96c9ca01c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:59.126449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:59.169448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:53:59.213735 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7219d0d-a67e-4801-b4bb-7bb95a26291f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:53:59.215251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f7219d0d-a67e-4801-b4bb-7bb95a26291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:59.360235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7219d0d-a67e-4801-b4bb-7bb95a26291f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:53:59.434580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:53:59.475965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042caf8e-0075-4087-83a6-f597d306ba45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:53:59.476621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042caf8e-0075-4087-83a6-f597d306ba45 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:53:59.476944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042caf8e-0075-4087-83a6-f597d306ba45 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:53:59.477226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042caf8e-0075-4087-83a6-f597d306ba45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:53:59.478893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-042caf8e-0075-4087-83a6-f597d306ba45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:53:59.567769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042caf8e-0075-4087-83a6-f597d306ba45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:53:59.569976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042caf8e-0075-4087-83a6-f597d306ba45 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:53:59.588369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 707/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:00.514363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100faa72-398e-4e44-9051-a5d052d435db None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:00.514897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100faa72-398e-4e44-9051-a5d052d435db None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:00.514990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100faa72-398e-4e44-9051-a5d052d435db None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:00.516306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:00.521206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fa2c0-f4b7-4bfd-b252-222987207f72 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:00.521619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fa2c0-f4b7-4bfd-b252-222987207f72 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:00.521772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fa2c0-f4b7-4bfd-b252-222987207f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:00.600627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a03fa2c0-f4b7-4bfd-b252-222987207f72 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:00.602026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a03fa2c0-f4b7-4bfd-b252-222987207f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:00.631145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:00.701279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:00.857469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100faa72-398e-4e44-9051-a5d052d435db None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:00.859079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-100faa72-398e-4e44-9051-a5d052d435db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:01.020079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100faa72-398e-4e44-9051-a5d052d435db None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:01.070018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:01.156934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a1eed-1b83-4036-8d5e-b123f2651507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:01.157604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a1eed-1b83-4036-8d5e-b123f2651507 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:01.157894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a1eed-1b83-4036-8d5e-b123f2651507 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:01.158167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a1eed-1b83-4036-8d5e-b123f2651507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:01.159832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-487a1eed-1b83-4036-8d5e-b123f2651507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:01.259663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a1eed-1b83-4036-8d5e-b123f2651507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:01.261329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a1eed-1b83-4036-8d5e-b123f2651507 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:01.280880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 711/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:02.203317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28146-f709-4f40-b4cf-0e62ede9790d None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:02.203957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28146-f709-4f40-b4cf-0e62ede9790d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:02.203957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28146-f709-4f40-b4cf-0e62ede9790d None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:02.205314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:02.210323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2c8c2-2f70-4d42-86bf-8baa82aa4066 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:02.210785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2c8c2-2f70-4d42-86bf-8baa82aa4066 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:02.210882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2c8c2-2f70-4d42-86bf-8baa82aa4066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:02.263871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:02.285218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8cb2c8c2-2f70-4d42-86bf-8baa82aa4066 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:02.286405 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8cb2c8c2-2f70-4d42-86bf-8baa82aa4066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:02.382916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:02.532951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28146-f709-4f40-b4cf-0e62ede9790d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:02.534459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-05f28146-f709-4f40-b4cf-0e62ede9790d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:02.681046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28146-f709-4f40-b4cf-0e62ede9790d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:02.693271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:02.796922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3396d-7f03-4116-abcd-f8ce202a37b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:02.796922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3396d-7f03-4116-abcd-f8ce202a37b8 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:02.796922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3396d-7f03-4116-abcd-f8ce202a37b8 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:02.796922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3396d-7f03-4116-abcd-f8ce202a37b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:02.797645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-18d3396d-7f03-4116-abcd-f8ce202a37b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:02.886734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3396d-7f03-4116-abcd-f8ce202a37b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:02.888338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3396d-7f03-4116-abcd-f8ce202a37b8 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:02.906397 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 710/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:03.771588 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13954926-e2b1-45de-a16d-6013ebfb4841 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:03.772159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13954926-e2b1-45de-a16d-6013ebfb4841 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:03.772159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13954926-e2b1-45de-a16d-6013ebfb4841 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:03.773550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:03.778819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee2a6f-ac93-4856-8f6a-9662151086e6 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:03.779235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee2a6f-ac93-4856-8f6a-9662151086e6 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:03.779377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee2a6f-ac93-4856-8f6a-9662151086e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:03.798414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:03.855098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-18ee2a6f-ac93-4856-8f6a-9662151086e6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:03.856342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-18ee2a6f-ac93-4856-8f6a-9662151086e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:03.951310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:04.101513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13954926-e2b1-45de-a16d-6013ebfb4841 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:04.102946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-13954926-e2b1-45de-a16d-6013ebfb4841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:04.227317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:04.269004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13954926-e2b1-45de-a16d-6013ebfb4841 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:04.400749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79ad5c-5ba7-412a-a818-caaa05c98bd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:04.401325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79ad5c-5ba7-412a-a818-caaa05c98bd8 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:04.401611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79ad5c-5ba7-412a-a818-caaa05c98bd8 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:04.401888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79ad5c-5ba7-412a-a818-caaa05c98bd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:04.403578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7e79ad5c-5ba7-412a-a818-caaa05c98bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:04.491701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:04.506910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79ad5c-5ba7-412a-a818-caaa05c98bd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:04.508560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79ad5c-5ba7-412a-a818-caaa05c98bd8 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:04.529047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 714/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:05.434058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9a374-ef2b-410d-88fb-ad8b004282be None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:05.434518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9a374-ef2b-410d-88fb-ad8b004282be None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:05.434636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9a374-ef2b-410d-88fb-ad8b004282be None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:05.435999 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:05.441053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397371-c941-4bfc-a4f6-c84fe5616e46 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:05.441515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397371-c941-4bfc-a4f6-c84fe5616e46 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:05.441612 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397371-c941-4bfc-a4f6-c84fe5616e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:05.521797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ee397371-c941-4bfc-a4f6-c84fe5616e46 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:05.523018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ee397371-c941-4bfc-a4f6-c84fe5616e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:05.649538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:05.801943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9a374-ef2b-410d-88fb-ad8b004282be None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:05.804112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e2a9a374-ef2b-410d-88fb-ad8b004282be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:05.888654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:05.956690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9a374-ef2b-410d-88fb-ad8b004282be None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:06.076155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e95a3a-71b1-46e1-a6f9-bf9230ceabe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:06.076840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e95a3a-71b1-46e1-a6f9-bf9230ceabe6 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:06.077131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e95a3a-71b1-46e1-a6f9-bf9230ceabe6 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:06.077410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e95a3a-71b1-46e1-a6f9-bf9230ceabe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:06.079072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e5e95a3a-71b1-46e1-a6f9-bf9230ceabe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:06.133067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:06.173406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e95a3a-71b1-46e1-a6f9-bf9230ceabe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:06.175022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e95a3a-71b1-46e1-a6f9-bf9230ceabe6 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:06.204589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 713/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:07.147000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79165278-ef20-46af-965b-12419cc748a1 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:07.147529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79165278-ef20-46af-965b-12419cc748a1 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:07.147529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79165278-ef20-46af-965b-12419cc748a1 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:07.149284 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:54:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:07.158387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e97a1-409e-4039-b98f-76c959478607 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:07.159290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e97a1-409e-4039-b98f-76c959478607 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:07.159633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e97a1-409e-4039-b98f-76c959478607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:07.238856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f41e97a1-409e-4039-b98f-76c959478607 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:07.240111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f41e97a1-409e-4039-b98f-76c959478607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:07.343437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:07.499719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79165278-ef20-46af-965b-12419cc748a1 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:07.503113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-79165278-ef20-46af-965b-12419cc748a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:07.552625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:07.652993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79165278-ef20-46af-965b-12419cc748a1 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:07.776217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ceca4-e47a-4ed8-8659-9fc8b6d891d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:07.776923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ceca4-e47a-4ed8-8659-9fc8b6d891d9 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:07.777202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ceca4-e47a-4ed8-8659-9fc8b6d891d9 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:07.777477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ceca4-e47a-4ed8-8659-9fc8b6d891d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:07.779241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-422ceca4-e47a-4ed8-8659-9fc8b6d891d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:07.788071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:07.879395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ceca4-e47a-4ed8-8659-9fc8b6d891d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:07.881735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ceca4-e47a-4ed8-8659-9fc8b6d891d9 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:07.899564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 717/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:08.821111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f652a77-83a8-4429-9e77-77f96e84a6f9 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:08.821646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f652a77-83a8-4429-9e77-77f96e84a6f9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:08.821646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f652a77-83a8-4429-9e77-77f96e84a6f9 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:08.823045 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:08.828184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574c9e1-0ea9-4f6f-9833-cd8b3cfd7d7a None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:08.832375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574c9e1-0ea9-4f6f-9833-cd8b3cfd7d7a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:08.832375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574c9e1-0ea9-4f6f-9833-cd8b3cfd7d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:08.903646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a574c9e1-0ea9-4f6f-9833-cd8b3cfd7d7a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:08.904881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a574c9e1-0ea9-4f6f-9833-cd8b3cfd7d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:08.975104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:09.003678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:09.153373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f652a77-83a8-4429-9e77-77f96e84a6f9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:09.154974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4f652a77-83a8-4429-9e77-77f96e84a6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:09.173590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:09.303640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f652a77-83a8-4429-9e77-77f96e84a6f9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:09.390421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:09.416351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d5eaa-a68a-46ef-ae1e-e942d02ea204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:09.417060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d5eaa-a68a-46ef-ae1e-e942d02ea204 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:09.417351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d5eaa-a68a-46ef-ae1e-e942d02ea204 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:09.417631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d5eaa-a68a-46ef-ae1e-e942d02ea204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:09.419284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-cf4d5eaa-a68a-46ef-ae1e-e942d02ea204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:09.508833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d5eaa-a68a-46ef-ae1e-e942d02ea204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:09.510449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d5eaa-a68a-46ef-ae1e-e942d02ea204 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:09.528847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 716/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:10.419100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf9a7-2974-47c5-81bb-0d5f9450b5a0 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:10.419860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf9a7-2974-47c5-81bb-0d5f9450b5a0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:10.419860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf9a7-2974-47c5-81bb-0d5f9450b5a0 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:10.421143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:10.427490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e46f22-eb77-4a2f-9e28-37142dca6f66 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:10.427914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e46f22-eb77-4a2f-9e28-37142dca6f66 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:10.428091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e46f22-eb77-4a2f-9e28-37142dca6f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:10.503273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-93e46f22-eb77-4a2f-9e28-37142dca6f66 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:10.504493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-93e46f22-eb77-4a2f-9e28-37142dca6f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:10.546120 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:10.601616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:10.756028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bf9a7-2974-47c5-81bb-0d5f9450b5a0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:10.757581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7e4bf9a7-2974-47c5-81bb-0d5f9450b5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:10.903307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bf9a7-2974-47c5-81bb-0d5f9450b5a0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:10.959017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:11.018944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b495b-2ed1-4ab6-9d6f-c784283317da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:11.019537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b495b-2ed1-4ab6-9d6f-c784283317da service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:11.019813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b495b-2ed1-4ab6-9d6f-c784283317da service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:11.020099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b495b-2ed1-4ab6-9d6f-c784283317da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:11.022445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-799b495b-2ed1-4ab6-9d6f-c784283317da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:11.112208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b495b-2ed1-4ab6-9d6f-c784283317da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:11.113842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b495b-2ed1-4ab6-9d6f-c784283317da service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:11.132330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 720/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:12.013032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9e6c-6c90-4a37-85c0-460c8c16a718 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:12.013570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9e6c-6c90-4a37-85c0-460c8c16a718 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:12.013706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9e6c-6c90-4a37-85c0-460c8c16a718 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:12.014964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:12.019863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107a610-1e0b-4138-a88c-135371e27512 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:12.020316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107a610-1e0b-4138-a88c-135371e27512 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:12.020429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107a610-1e0b-4138-a88c-135371e27512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:12.095053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8107a610-1e0b-4138-a88c-135371e27512 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:12.096252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8107a610-1e0b-4138-a88c-135371e27512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:12.098282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:12.199360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:12.351784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d9e6c-6c90-4a37-85c0-460c8c16a718 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:12.353234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8c5d9e6c-6c90-4a37-85c0-460c8c16a718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:12.498699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d9e6c-6c90-4a37-85c0-460c8c16a718 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:12.526119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:12.612334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa50e9-f28e-4d19-8b58-832a789149ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:12.612934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa50e9-f28e-4d19-8b58-832a789149ab service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:12.613228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa50e9-f28e-4d19-8b58-832a789149ab service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:12.613509 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa50e9-f28e-4d19-8b58-832a789149ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:12.615261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9daa50e9-f28e-4d19-8b58-832a789149ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:12.828236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa50e9-f28e-4d19-8b58-832a789149ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:12.829863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa50e9-f28e-4d19-8b58-832a789149ab service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:12.847836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 719/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:14.797917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178d67-c448-4d94-b689-a7527c6fcdb6 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:14.798429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178d67-c448-4d94-b689-a7527c6fcdb6 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:14.798541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178d67-c448-4d94-b689-a7527c6fcdb6 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:14.799895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:14.806236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a2e0c-99c0-473a-b547-99e3c94a169b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:14.806646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a2e0c-99c0-473a-b547-99e3c94a169b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:14.806790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a2e0c-99c0-473a-b547-99e3c94a169b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:14.847355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:14.871554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-620a2e0c-99c0-473a-b547-99e3c94a169b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:14.872789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-620a2e0c-99c0-473a-b547-99e3c94a169b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:14.968983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:15.009116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178d67-c448-4d94-b689-a7527c6fcdb6 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:15.010632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-62178d67-c448-4d94-b689-a7527c6fcdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:15.115950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178d67-c448-4d94-b689-a7527c6fcdb6 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:15.147793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:15.190146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29539fe8-ed92-42b1-919d-e0b447c3c30e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:15.190869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29539fe8-ed92-42b1-919d-e0b447c3c30e service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:15.191153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29539fe8-ed92-42b1-919d-e0b447c3c30e service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:15.191429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29539fe8-ed92-42b1-919d-e0b447c3c30e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:15.193772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-29539fe8-ed92-42b1-919d-e0b447c3c30e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:15.299647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29539fe8-ed92-42b1-919d-e0b447c3c30e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:15.301364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29539fe8-ed92-42b1-919d-e0b447c3c30e service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:15.321431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 723/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:17.610815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3da9e-f965-409d-985b-ffbf128e5bf9 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:17.611368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3da9e-f965-409d-985b-ffbf128e5bf9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:17.611368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3da9e-f965-409d-985b-ffbf128e5bf9 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:17.612783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:17.618509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac66272-7f3a-4a97-9cb7-2610e2dba871 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:17.618965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac66272-7f3a-4a97-9cb7-2610e2dba871 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:17.619080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac66272-7f3a-4a97-9cb7-2610e2dba871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:17.685177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-9ac66272-7f3a-4a97-9cb7-2610e2dba871 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:17.686312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9ac66272-7f3a-4a97-9cb7-2610e2dba871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:17.699900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:17.788448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:17.826762 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3da9e-f965-409d-985b-ffbf128e5bf9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:17.828281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-52c3da9e-f965-409d-985b-ffbf128e5bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:17.929890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3da9e-f965-409d-985b-ffbf128e5bf9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:17.943146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:18.003321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7433d3-5783-4e6a-9b43-5374cddb0163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:18.003970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7433d3-5783-4e6a-9b43-5374cddb0163 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:18.004294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7433d3-5783-4e6a-9b43-5374cddb0163 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:18.004597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7433d3-5783-4e6a-9b43-5374cddb0163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:18.006809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fa7433d3-5783-4e6a-9b43-5374cddb0163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:18.115327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7433d3-5783-4e6a-9b43-5374cddb0163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:18.116816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7433d3-5783-4e6a-9b43-5374cddb0163 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:18.135424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 722/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:20.583556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70ab6e-45ba-4385-b881-2055a5e91ffd None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:20.584117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70ab6e-45ba-4385-b881-2055a5e91ffd None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:20.584117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70ab6e-45ba-4385-b881-2055a5e91ffd None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:20.585503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:20.590708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990736b8-f8a1-4037-90f4-a32013f86d13 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:20.591166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990736b8-f8a1-4037-90f4-a32013f86d13 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:20.591291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990736b8-f8a1-4037-90f4-a32013f86d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:20.638579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:20.656238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-990736b8-f8a1-4037-90f4-a32013f86d13 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:20.657481 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-990736b8-f8a1-4037-90f4-a32013f86d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:20.756607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:20.795468 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70ab6e-45ba-4385-b881-2055a5e91ffd None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:20.797018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3b70ab6e-45ba-4385-b881-2055a5e91ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:20.903727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70ab6e-45ba-4385-b881-2055a5e91ffd None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:20.943836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:20.978218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fbf58-ba8f-421f-83c9-db42f7f33066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:20.978908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fbf58-ba8f-421f-83c9-db42f7f33066 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:20.979197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fbf58-ba8f-421f-83c9-db42f7f33066 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:20.979596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fbf58-ba8f-421f-83c9-db42f7f33066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:20.981374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7e0fbf58-ba8f-421f-83c9-db42f7f33066 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:21.089963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fbf58-ba8f-421f-83c9-db42f7f33066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:21.091315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fbf58-ba8f-421f-83c9-db42f7f33066 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:21.109158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 726/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:23.335543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82c65a-6da3-4b76-9395-e8c5e62af86b None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:23.336085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82c65a-6da3-4b76-9395-e8c5e62af86b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:23.336085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82c65a-6da3-4b76-9395-e8c5e62af86b None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:23.337485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:23.342966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60845e-83a4-4c68-ad99-0bdc9bf6f522 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:23.343389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60845e-83a4-4c68-ad99-0bdc9bf6f522 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:23.343532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60845e-83a4-4c68-ad99-0bdc9bf6f522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:23.407818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-4d60845e-83a4-4c68-ad99-0bdc9bf6f522 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:23.409074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4d60845e-83a4-4c68-ad99-0bdc9bf6f522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:23.431445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:23.508371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:23.549290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82c65a-6da3-4b76-9395-e8c5e62af86b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:23.551109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-cf82c65a-6da3-4b76-9395-e8c5e62af86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:23.663731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82c65a-6da3-4b76-9395-e8c5e62af86b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:23.683287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:23.737130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce9a88-bb02-4abb-a2f9-25b8f7bdc39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:23.737788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9a88-bb02-4abb-a2f9-25b8f7bdc39d service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:23.738072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9a88-bb02-4abb-a2f9-25b8f7bdc39d service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:23.738345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9a88-bb02-4abb-a2f9-25b8f7bdc39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:23.740030 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-45ce9a88-bb02-4abb-a2f9-25b8f7bdc39d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:23.850909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce9a88-bb02-4abb-a2f9-25b8f7bdc39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:23.852360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce9a88-bb02-4abb-a2f9-25b8f7bdc39d service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:23.871740 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 725/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:25.957383 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f429a05-5734-4154-b4ae-8d806069547b None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:25.959674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f429a05-5734-4154-b4ae-8d806069547b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:25.959674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f429a05-5734-4154-b4ae-8d806069547b None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:25.959674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:25.965099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfea0d-ddf6-4f88-93bc-0464c32e4a17 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:25.965557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfea0d-ddf6-4f88-93bc-0464c32e4a17 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:25.965657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfea0d-ddf6-4f88-93bc-0464c32e4a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:26.028859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:26.031814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-cfbfea0d-ddf6-4f88-93bc-0464c32e4a17 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:26.033059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-cfbfea0d-ddf6-4f88-93bc-0464c32e4a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:26.127448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:26.165713 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f429a05-5734-4154-b4ae-8d806069547b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:26.167348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1f429a05-5734-4154-b4ae-8d806069547b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:26.281540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f429a05-5734-4154-b4ae-8d806069547b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:26.330311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:26.356887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f6f0a-3776-49fe-8fe1-4c305e377bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:26.357539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f6f0a-3776-49fe-8fe1-4c305e377bce service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:26.357859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f6f0a-3776-49fe-8fe1-4c305e377bce service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:26.358137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f6f0a-3776-49fe-8fe1-4c305e377bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:26.359765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-576f6f0a-3776-49fe-8fe1-4c305e377bce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:26.481820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f6f0a-3776-49fe-8fe1-4c305e377bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:26.483260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f6f0a-3776-49fe-8fe1-4c305e377bce service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:26.502459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 729/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:28.630923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce580c3f-f857-426d-800e-a35af5de5464 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:28.631490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce580c3f-f857-426d-800e-a35af5de5464 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:28.631490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce580c3f-f857-426d-800e-a35af5de5464 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:28.632919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:28.638306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fb427-eaf4-486f-b19d-3076990f529f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:28.638734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fb427-eaf4-486f-b19d-3076990f529f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:28.639206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fb427-eaf4-486f-b19d-3076990f529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:28.705678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-914fb427-eaf4-486f-b19d-3076990f529f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:28.706819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-914fb427-eaf4-486f-b19d-3076990f529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:28.736078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:28.807680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:28.846950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce580c3f-f857-426d-800e-a35af5de5464 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:28.848579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ce580c3f-f857-426d-800e-a35af5de5464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:28.952260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce580c3f-f857-426d-800e-a35af5de5464 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:28.982767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:29.029720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82410f2d-15d4-4c72-941d-0855366f2954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:29.030368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82410f2d-15d4-4c72-941d-0855366f2954 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:29.030649 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82410f2d-15d4-4c72-941d-0855366f2954 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:29.030921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82410f2d-15d4-4c72-941d-0855366f2954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:29.032540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-82410f2d-15d4-4c72-941d-0855366f2954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:29.140325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82410f2d-15d4-4c72-941d-0855366f2954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:29.141796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82410f2d-15d4-4c72-941d-0855366f2954 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:29.161396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 728/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:31.293397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a8883-a59d-4979-a0e0-430d4b2dbe0e None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:31.293972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a8883-a59d-4979-a0e0-430d4b2dbe0e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:31.293972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a8883-a59d-4979-a0e0-430d4b2dbe0e None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:31.295374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:31.300881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52784f2f-e984-4e64-97fd-cf204a3fe16c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:31.301261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52784f2f-e984-4e64-97fd-cf204a3fe16c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:31.301531 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52784f2f-e984-4e64-97fd-cf204a3fe16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:31.371295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-52784f2f-e984-4e64-97fd-cf204a3fe16c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:31.372520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-52784f2f-e984-4e64-97fd-cf204a3fe16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:31.381367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:31.470495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:31.508250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a8883-a59d-4979-a0e0-430d4b2dbe0e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:31.509833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2b2a8883-a59d-4979-a0e0-430d4b2dbe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:31.615323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a8883-a59d-4979-a0e0-430d4b2dbe0e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:31.681014 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:31.700280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330dc6c-af6a-4ab5-a44a-a72601d3f6cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:31.701078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330dc6c-af6a-4ab5-a44a-a72601d3f6cc service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:31.701313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330dc6c-af6a-4ab5-a44a-a72601d3f6cc service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:31.701590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330dc6c-af6a-4ab5-a44a-a72601d3f6cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:31.703233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d330dc6c-af6a-4ab5-a44a-a72601d3f6cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:31.812122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330dc6c-af6a-4ab5-a44a-a72601d3f6cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:31.813579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330dc6c-af6a-4ab5-a44a-a72601d3f6cc service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:31.832659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 732/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:33.895691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909d73c-d256-40f9-a47c-483dc84de642 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:33.896259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909d73c-d256-40f9-a47c-483dc84de642 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:33.896259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909d73c-d256-40f9-a47c-483dc84de642 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:33.897655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:33.902809 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd1d54-fe1a-4aa7-8be6-d2de99ff4de6 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:33.903234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd1d54-fe1a-4aa7-8be6-d2de99ff4de6 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:33.903371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd1d54-fe1a-4aa7-8be6-d2de99ff4de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:33.972183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-17cd1d54-fe1a-4aa7-8be6-d2de99ff4de6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:33.973493 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-17cd1d54-fe1a-4aa7-8be6-d2de99ff4de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:34.013450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:34.076089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:34.113990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909d73c-d256-40f9-a47c-483dc84de642 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:34.116225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2909d73c-d256-40f9-a47c-483dc84de642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:34.220749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909d73c-d256-40f9-a47c-483dc84de642 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:34.258099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:34.294136 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f7f32-b9b1-4628-8cf2-4f9349e989f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:34.294789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f7f32-b9b1-4628-8cf2-4f9349e989f2 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:34.295073 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f7f32-b9b1-4628-8cf2-4f9349e989f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:34.295346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f7f32-b9b1-4628-8cf2-4f9349e989f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:34.297721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-826f7f32-b9b1-4628-8cf2-4f9349e989f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:34.403307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f7f32-b9b1-4628-8cf2-4f9349e989f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:34.404803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f7f32-b9b1-4628-8cf2-4f9349e989f2 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:34.423606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 731/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:36.485508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965927-b6dd-476e-9d7a-a19a248c5172 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:36.486066 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965927-b6dd-476e-9d7a-a19a248c5172 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:36.486066 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965927-b6dd-476e-9d7a-a19a248c5172 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:36.487470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:36.493326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f92c0-897f-4224-8e61-8579c46a0193 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:36.493735 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f92c0-897f-4224-8e61-8579c46a0193 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:36.493886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f92c0-897f-4224-8e61-8579c46a0193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:36.559013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-120f92c0-897f-4224-8e61-8579c46a0193 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:36.560280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-120f92c0-897f-4224-8e61-8579c46a0193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:36.578530 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:36.664063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:54:36.701604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965927-b6dd-476e-9d7a-a19a248c5172 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:36.703126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ce965927-b6dd-476e-9d7a-a19a248c5172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:36.808793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965927-b6dd-476e-9d7a-a19a248c5172 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:36.876937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:36.883802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355fad3-1546-4a48-ab49-949e51d4e9d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:36.884457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355fad3-1546-4a48-ab49-949e51d4e9d5 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:36.884778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355fad3-1546-4a48-ab49-949e51d4e9d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:36.885058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355fad3-1546-4a48-ab49-949e51d4e9d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:36.886709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7355fad3-1546-4a48-ab49-949e51d4e9d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:37.013042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355fad3-1546-4a48-ab49-949e51d4e9d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:37.014477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355fad3-1546-4a48-ab49-949e51d4e9d5 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:37.038829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 735/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:39.111937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6c567-7dc7-4754-a123-734b67e9c2f7 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:39.112441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6c567-7dc7-4754-a123-734b67e9c2f7 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:39.112441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6c567-7dc7-4754-a123-734b67e9c2f7 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:39.114410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:54:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:39.119880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e3cfd-e3f5-4c6d-8432-5621168ed6f3 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:39.120286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e3cfd-e3f5-4c6d-8432-5621168ed6f3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:39.123303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e3cfd-e3f5-4c6d-8432-5621168ed6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:39.187641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-793e3cfd-e3f5-4c6d-8432-5621168ed6f3 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:39.188864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-793e3cfd-e3f5-4c6d-8432-5621168ed6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:39.238284 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:39.286760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:39.323975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6c567-7dc7-4754-a123-734b67e9c2f7 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:39.325673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d9e6c567-7dc7-4754-a123-734b67e9c2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:39.450734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6c567-7dc7-4754-a123-734b67e9c2f7 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:54:39.498087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:39.525064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad2f57-26d6-4080-9319-319704e954bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:54:39.525716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad2f57-26d6-4080-9319-319704e954bc service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:39.526003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad2f57-26d6-4080-9319-319704e954bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:39.526309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad2f57-26d6-4080-9319-319704e954bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:39.527948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e9ad2f57-26d6-4080-9319-319704e954bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:39.648133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad2f57-26d6-4080-9319-319704e954bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:54:39.649827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad2f57-26d6-4080-9319-319704e954bc service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:54:39.670426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 734/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:54:49.284234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=22 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:49.667584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=66 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:59.576906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae485111-c58b-4421-ada2-4b364bf23e49 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:59.577459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae485111-c58b-4421-ada2-4b364bf23e49 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:59.577459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae485111-c58b-4421-ada2-4b364bf23e49 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:59.578843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 737/1471] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:59.584259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07061da7-478d-476f-b119-527998915af8 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:59.584676 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07061da7-478d-476f-b119-527998915af8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:59.584857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07061da7-478d-476f-b119-527998915af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:59.650028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-07061da7-478d-476f-b119-527998915af8 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:59.651261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-07061da7-478d-476f-b119-527998915af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:59.748635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 735/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:54:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:59.833383 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22954-5aa5-4450-9097-4107cc6d4a3c None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:59.833672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22954-5aa5-4450-9097-4107cc6d4a3c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:59.833951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22954-5aa5-4450-9097-4107cc6d4a3c None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:59.835341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 738/1473] 10.222.0.17 () {54 vars in 966 bytes} [Sun Jul 24 19:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:54:59.840089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337d9ec-1e29-42c7-b55d-47ff93ff11c0 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:54:59.840380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337d9ec-1e29-42c7-b55d-47ff93ff11c0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:54:59.840688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337d9ec-1e29-42c7-b55d-47ff93ff11c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:54:59.913863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2337d9ec-1e29-42c7-b55d-47ff93ff11c0 None None] MFA Rules not processed for user `bca33ef49ba44832a7fb45b2317d211c`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:54:59.915090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2337d9ec-1e29-42c7-b55d-47ff93ff11c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:54:59.935835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:54:59.990451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 736/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Sun Jul 24 19:54:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:54:59.997958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede22954-5aa5-4450-9097-4107cc6d4a3c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:54:59.999526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ede22954-5aa5-4450-9097-4107cc6d4a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:00.102718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede22954-5aa5-4450-9097-4107cc6d4a3c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:55:00.176282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d5520-dfc7-414e-b161-da6eb80e5a25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca33ef49ba44832a7fb45b2317d211c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:55:00.176966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d5520-dfc7-414e-b161-da6eb80e5a25 service glance] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:00.177247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d5520-dfc7-414e-b161-da6eb80e5a25 service glance] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:00.177520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d5520-dfc7-414e-b161-da6eb80e5a25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:00.179160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0d0d5520-dfc7-414e-b161-da6eb80e5a25 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:00.233916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:00.295612 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d5520-dfc7-414e-b161-da6eb80e5a25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:55:00.297420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d5520-dfc7-414e-b161-da6eb80e5a25 service glance] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:55:00.317533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 739/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:54:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:55:02.459655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d7580-fa03-47a2-9533-658cef18780d None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:02.460151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d7580-fa03-47a2-9533-658cef18780d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:02.460256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d7580-fa03-47a2-9533-658cef18780d None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:02.461621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 737/1476] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:55:02.466950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc04af-d9f7-416e-b750-8ddde69cfc41 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:02.467383 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc04af-d9f7-416e-b750-8ddde69cfc41 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:02.467495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc04af-d9f7-416e-b750-8ddde69cfc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:02.535433 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f9cc04af-d9f7-416e-b750-8ddde69cfc41 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:55:02.536799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f9cc04af-d9f7-416e-b750-8ddde69cfc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:02.641000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 740/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:55:02.665567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d7580-fa03-47a2-9533-658cef18780d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:55:02.667110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d60d7580-fa03-47a2-9533-658cef18780d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:02.808515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d7580-fa03-47a2-9533-658cef18780d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:55:02.872823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:02.920610 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7f98a-3388-4dbb-850e-75c067bd081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:55:02.921305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7f98a-3388-4dbb-850e-75c067bd081f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:02.921589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7f98a-3388-4dbb-850e-75c067bd081f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:02.921863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7f98a-3388-4dbb-850e-75c067bd081f admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:02.922638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7f98a-3388-4dbb-850e-75c067bd081f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:55:02.925353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING py.warnings [None req-8bc7f98a-3388-4dbb-850e-75c067bd081f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 19:55:02.925353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: warnings.warn(msg) Jul 24 19:55:02.925353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: Jul 24 19:55:02.926962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7f98a-3388-4dbb-850e-75c067bd081f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:55:02.948413 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 738/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Jul 24 19:55:02 2022] GET /identity/v3/projects => generated 2553 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 19:55:05.068970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8793-4025-4e03-825a-a750fe203175 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:05.069494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8793-4025-4e03-825a-a750fe203175 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:05.069625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8793-4025-4e03-825a-a750fe203175 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:05.071709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 741/1479] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:55:05.077340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d487aee-6e57-45ea-afc9-0548e03daff4 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:05.077802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d487aee-6e57-45ea-afc9-0548e03daff4 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:05.077915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d487aee-6e57-45ea-afc9-0548e03daff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:05.149441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-5d487aee-6e57-45ea-afc9-0548e03daff4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:55:05.150663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5d487aee-6e57-45ea-afc9-0548e03daff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:05.248330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 739/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:55:05.292912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa8793-4025-4e03-825a-a750fe203175 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:55:05.294428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-defa8793-4025-4e03-825a-a750fe203175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:05.305212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:05.444501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa8793-4025-4e03-825a-a750fe203175 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:55:05.522522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:05.554791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74a6a-8e30-4c75-a644-7909d00f51da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:55:05.555486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74a6a-8e30-4c75-a644-7909d00f51da service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:05.555715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74a6a-8e30-4c75-a644-7909d00f51da service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:05.555986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74a6a-8e30-4c75-a644-7909d00f51da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:05.557646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0ba74a6a-8e30-4c75-a644-7909d00f51da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:05.647504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba74a6a-8e30-4c75-a644-7909d00f51da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:55:05.649145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba74a6a-8e30-4c75-a644-7909d00f51da service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:55:05.667163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 742/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:55:07.922317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1419c6b-b9ca-49ba-9549-50ea07f52be7 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:07.922886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1419c6b-b9ca-49ba-9549-50ea07f52be7 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:07.922886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1419c6b-b9ca-49ba-9549-50ea07f52be7 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:07.924236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 740/1482] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:55:07.930502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83c85-ff03-4b88-baf8-bb57a3ce7e30 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:07.930965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83c85-ff03-4b88-baf8-bb57a3ce7e30 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:07.930965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83c85-ff03-4b88-baf8-bb57a3ce7e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:07.998168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8fa83c85-ff03-4b88-baf8-bb57a3ce7e30 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:55:07.999379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8fa83c85-ff03-4b88-baf8-bb57a3ce7e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:08.045088 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:08.095054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 743/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:55:08.132935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1419c6b-b9ca-49ba-9549-50ea07f52be7 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:55:08.134485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f1419c6b-b9ca-49ba-9549-50ea07f52be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:08.158713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:08.238112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1419c6b-b9ca-49ba-9549-50ea07f52be7 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:55:08.316528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772a5bd-463a-442a-960c-a26a54d9983e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:55:08.317253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772a5bd-463a-442a-960c-a26a54d9983e service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:08.317542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772a5bd-463a-442a-960c-a26a54d9983e service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:08.317841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772a5bd-463a-442a-960c-a26a54d9983e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:08.319467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2772a5bd-463a-442a-960c-a26a54d9983e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:08.395094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:08.431079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772a5bd-463a-442a-960c-a26a54d9983e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:55:08.432663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772a5bd-463a-442a-960c-a26a54d9983e service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:55:08.451092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 741/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 19:55:10.579222 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380afd8-8d35-483f-af0a-841b77056ab9 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:10.579788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380afd8-8d35-483f-af0a-841b77056ab9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:10.579788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380afd8-8d35-483f-af0a-841b77056ab9 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:10.581240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 744/1485] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:55:10.586597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116eac9e-d101-457c-8aea-e70a75cf72c4 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:10.587029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116eac9e-d101-457c-8aea-e70a75cf72c4 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:10.587135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116eac9e-d101-457c-8aea-e70a75cf72c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:10.656237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-116eac9e-d101-457c-8aea-e70a75cf72c4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:55:10.657398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-116eac9e-d101-457c-8aea-e70a75cf72c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:10.754891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 742/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:55:10.794514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380afd8-8d35-483f-af0a-841b77056ab9 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:55:10.796106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2380afd8-8d35-483f-af0a-841b77056ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:10.902165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380afd8-8d35-483f-af0a-841b77056ab9 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:55:10.979686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:10.981441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585e84c-10f9-4541-8341-da96106e3f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:55:10.982088 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585e84c-10f9-4541-8341-da96106e3f44 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:10.982370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585e84c-10f9-4541-8341-da96106e3f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:10.982642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585e84c-10f9-4541-8341-da96106e3f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:10.984288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3585e84c-10f9-4541-8341-da96106e3f44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:11.093428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585e84c-10f9-4541-8341-da96106e3f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:55:11.094917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585e84c-10f9-4541-8341-da96106e3f44 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:55:11.112939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 745/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:55:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:55:13.298827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f2eb5-c825-4f44-b409-6e06d2539908 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:13.299441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f2eb5-c825-4f44-b409-6e06d2539908 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:13.299441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f2eb5-c825-4f44-b409-6e06d2539908 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:13.301406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 743/1488] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:55:13.308487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262f28-8be3-4eac-93fd-a3a567c798a2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:13.308989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262f28-8be3-4eac-93fd-a3a567c798a2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:13.309092 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262f28-8be3-4eac-93fd-a3a567c798a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:13.374098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-db262f28-8be3-4eac-93fd-a3a567c798a2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:55:13.375308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-db262f28-8be3-4eac-93fd-a3a567c798a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:13.429519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:13.475696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 746/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:55:13.518208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f2eb5-c825-4f44-b409-6e06d2539908 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:55:13.519791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-289f2eb5-c825-4f44-b409-6e06d2539908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:13.556111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:13.624419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f2eb5-c825-4f44-b409-6e06d2539908 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:55:13.698766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb076284-fdbf-4f40-ab77-3727ab220c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:55:13.699418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb076284-fdbf-4f40-ab77-3727ab220c38 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:13.699704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb076284-fdbf-4f40-ab77-3727ab220c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:13.699980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb076284-fdbf-4f40-ab77-3727ab220c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:13.701661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-eb076284-fdbf-4f40-ab77-3727ab220c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:13.789679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:13.820165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb076284-fdbf-4f40-ab77-3727ab220c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:55:13.821585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb076284-fdbf-4f40-ab77-3727ab220c38 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:55:13.839565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 744/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:55:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:55:16.336687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5578d-e9c9-4925-b6fe-d5300ad96989 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:16.337217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5578d-e9c9-4925-b6fe-d5300ad96989 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:16.337217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5578d-e9c9-4925-b6fe-d5300ad96989 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:16.338539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 747/1491] 10.222.0.17 () {54 vars in 926 bytes} [Sun Jul 24 19:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:55:16.344282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1c004-690d-4a60-abf4-865c543d8927 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:16.344845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1c004-690d-4a60-abf4-865c543d8927 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:16.344956 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1c004-690d-4a60-abf4-865c543d8927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:16.425373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-44b1c004-690d-4a60-abf4-865c543d8927 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:55:16.426618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-44b1c004-690d-4a60-abf4-865c543d8927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:16.536803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 745/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Sun Jul 24 19:55:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:55:16.577583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5578d-e9c9-4925-b6fe-d5300ad96989 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:55:16.579622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9bf5578d-e9c9-4925-b6fe-d5300ad96989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:16.691541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5578d-e9c9-4925-b6fe-d5300ad96989 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:55:16.773299 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ee533-18d9-4f49-ac87-d7fe799a020b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:55:16.773967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ee533-18d9-4f49-ac87-d7fe799a020b service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:55:16.774254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ee533-18d9-4f49-ac87-d7fe799a020b service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:55:16.774530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ee533-18d9-4f49-ac87-d7fe799a020b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:55:16.776247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-181ee533-18d9-4f49-ac87-d7fe799a020b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:55:16.787088 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:16.893820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ee533-18d9-4f49-ac87-d7fe799a020b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:55:16.895902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ee533-18d9-4f49-ac87-d7fe799a020b service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:55:16.914090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 748/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:55:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:55:26.534235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=82 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:55:26.911676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=50 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:57:22.249864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d47c1-4b0a-48b9-a3d5-64647ebec16f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:22.250478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d47c1-4b0a-48b9-a3d5-64647ebec16f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:22.250478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d47c1-4b0a-48b9-a3d5-64647ebec16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:22.349888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-500d47c1-4b0a-48b9-a3d5-64647ebec16f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:57:22.351199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-500d47c1-4b0a-48b9-a3d5-64647ebec16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:57:22.483194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 746/1494] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 19:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:57:22.494103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7761e9a-1ae9-4085-b7cb-d73933e02902 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:22.494103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7761e9a-1ae9-4085-b7cb-d73933e02902 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:22.494103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7761e9a-1ae9-4085-b7cb-d73933e02902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:22.513145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f7761e9a-1ae9-4085-b7cb-d73933e02902 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:57:22.514244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f7761e9a-1ae9-4085-b7cb-d73933e02902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:57:22.523083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 749/1495] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 19:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:57:22.527844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d47c1-4b0a-48b9-a3d5-64647ebec16f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:22.537838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d47c1-4b0a-48b9-a3d5-64647ebec16f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:22.542424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:22.543089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:22.543385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:22.543648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:22.544381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 19:57:22.544818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:22.545749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING py.warnings [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 19:57:22.545749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: warnings.warn(msg) Jul 24 19:57:22.545749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: Jul 24 19:57:22.547183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:22.563734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 747/1496] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 19:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:57:22.574504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7761e9a-1ae9-4085-b7cb-d73933e02902 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:22.584888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7761e9a-1ae9-4085-b7cb-d73933e02902 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:22.589569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570e07e-bfa4-4d34-8de9-c7861c32d90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:22.590236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570e07e-bfa4-4d34-8de9-c7861c32d90c admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:22.590557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570e07e-bfa4-4d34-8de9-c7861c32d90c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:22.590828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570e07e-bfa4-4d34-8de9-c7861c32d90c admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:22.591535 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570e07e-bfa4-4d34-8de9-c7861c32d90c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:22.593400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WARNING py.warnings [None req-c570e07e-bfa4-4d34-8de9-c7861c32d90c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 19:57:22.593400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: warnings.warn(msg) Jul 24 19:57:22.593400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: Jul 24 19:57:22.594668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570e07e-bfa4-4d34-8de9-c7861c32d90c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:22.705624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 750/1497] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 19:57:22 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:57:22.709901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:22.723584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe500b6d-74b6-40e3-8e30-6f9573f83d14 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:22.784239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:57:22.788727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3888c4-7dd8-472e-b56d-c39b98595187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:22.789413 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3888c4-7dd8-472e-b56d-c39b98595187 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:22.789698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3888c4-7dd8-472e-b56d-c39b98595187 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:22.789967 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3888c4-7dd8-472e-b56d-c39b98595187 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:22.790665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3888c4-7dd8-472e-b56d-c39b98595187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:22.791780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING py.warnings [None req-6f3888c4-7dd8-472e-b56d-c39b98595187 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 19:57:22.791780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: warnings.warn(msg) Jul 24 19:57:22.791780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: Jul 24 19:57:22.793054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3888c4-7dd8-472e-b56d-c39b98595187 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:22.874774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 748/1498] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 19:57:22 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 19:57:22.879029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570e07e-bfa4-4d34-8de9-c7861c32d90c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:22.901169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570e07e-bfa4-4d34-8de9-c7861c32d90c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:22.906905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92f078-9c9b-42ee-ac05-ca9d9079a4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:22.907688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92f078-9c9b-42ee-ac05-ca9d9079a4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:22.907986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92f078-9c9b-42ee-ac05-ca9d9079a4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:22.908288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92f078-9c9b-42ee-ac05-ca9d9079a4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:22.909130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92f078-9c9b-42ee-ac05-ca9d9079a4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:22.910229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WARNING py.warnings [None req-3f92f078-9c9b-42ee-ac05-ca9d9079a4c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 19:57:22.910229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: warnings.warn(msg) Jul 24 19:57:22.910229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: Jul 24 19:57:22.911192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92f078-9c9b-42ee-ac05-ca9d9079a4c3 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:22.927501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 751/1499] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 19:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 19:57:22.931876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3888c4-7dd8-472e-b56d-c39b98595187 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:22.942969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3888c4-7dd8-472e-b56d-c39b98595187 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:22.948484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626a934-7b07-4e7c-91e6-8d2fe2d0bd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:22.949249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626a934-7b07-4e7c-91e6-8d2fe2d0bd1a admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:22.949539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626a934-7b07-4e7c-91e6-8d2fe2d0bd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:22.949839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626a934-7b07-4e7c-91e6-8d2fe2d0bd1a admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:22.950686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626a934-7b07-4e7c-91e6-8d2fe2d0bd1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:22.951757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING py.warnings [None req-7626a934-7b07-4e7c-91e6-8d2fe2d0bd1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 19:57:22.951757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: warnings.warn(msg) Jul 24 19:57:22.951757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: Jul 24 19:57:22.952790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626a934-7b07-4e7c-91e6-8d2fe2d0bd1a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:22.969169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 749/1500] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 19:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 19:57:22.973646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92f078-9c9b-42ee-ac05-ca9d9079a4c3 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:22.984761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92f078-9c9b-42ee-ac05-ca9d9079a4c3 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:22.989897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8b270-c1c6-440b-afd9-e31f775a4fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:22.990824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8b270-c1c6-440b-afd9-e31f775a4fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:22.991106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8b270-c1c6-440b-afd9-e31f775a4fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:22.991384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8b270-c1c6-440b-afd9-e31f775a4fd0 admin admin] PATH_INFO: `/v3/projects/e80d332f409e4f0aac4e3d56193ae95c/users/f95a2d31bda84b15808841a0c261bb59/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:23.007481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8b270-c1c6-440b-afd9-e31f775a4fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80d332f409e4f0aac4e3d56193ae95c, user_id=f95a2d31bda84b15808841a0c261bb59, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:23.008528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WARNING py.warnings [None req-e6d8b270-c1c6-440b-afd9-e31f775a4fd0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 19:57:23.008528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: warnings.warn(msg) Jul 24 19:57:23.008528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: Jul 24 19:57:23.010483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8b270-c1c6-440b-afd9-e31f775a4fd0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:23.099783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 752/1501] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 19:57:22 2022] PUT /identity/v3/projects/e80d332f409e4f0aac4e3d56193ae95c/users/f95a2d31bda84b15808841a0c261bb59/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:57:23.115373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ed4c6-0d6e-497e-b057-32d1ea709f00 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:23.115727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ed4c6-0d6e-497e-b057-32d1ea709f00 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:23.116036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ed4c6-0d6e-497e-b057-32d1ea709f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:23.151327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-881ed4c6-0d6e-497e-b057-32d1ea709f00 None None] MFA Rules not processed for user `f95a2d31bda84b15808841a0c261bb59`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:57:23.153295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-881ed4c6-0d6e-497e-b057-32d1ea709f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:57:23.239743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 750/1502] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 19:57:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:57:23.256507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1b5ce-f841-4e8e-9b50-ae000eea13b4 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:23.256507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1b5ce-f841-4e8e-9b50-ae000eea13b4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:23.256507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1b5ce-f841-4e8e-9b50-ae000eea13b4 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:23.256507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 753/1503] 10.222.0.17 () {54 vars in 964 bytes} [Sun Jul 24 19:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:57:23.263347 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d58a-cea3-4ead-89e5-b18edaa202ed None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:23.263850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d58a-cea3-4ead-89e5-b18edaa202ed None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:23.263942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d58a-cea3-4ead-89e5-b18edaa202ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:23.334511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1191d58a-cea3-4ead-89e5-b18edaa202ed None None] MFA Rules not processed for user `fe7a67bd815147afa3f186e95b004469`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:57:23.335800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1191d58a-cea3-4ead-89e5-b18edaa202ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:57:23.420620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 751/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Sun Jul 24 19:57:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:57:23.434259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1b5ce-f841-4e8e-9b50-ae000eea13b4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:23.451635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1b5ce-f841-4e8e-9b50-ae000eea13b4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:23.458182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ce168-c893-4673-9528-9fd5dc648134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:23.459215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ce168-c893-4673-9528-9fd5dc648134 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:23.459722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ce168-c893-4673-9528-9fd5dc648134 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:23.460251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ce168-c893-4673-9528-9fd5dc648134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:23.474029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ce168-c893-4673-9528-9fd5dc648134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:23.475667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ce168-c893-4673-9528-9fd5dc648134 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:23.485811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 754/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:57:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:57:23.537217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c9d5-1bc2-41b5-98ca-30b1e5f9a70d None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:23.537544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c9d5-1bc2-41b5-98ca-30b1e5f9a70d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:23.537918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c9d5-1bc2-41b5-98ca-30b1e5f9a70d None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:23.539133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 752/1506] 10.222.0.17 () {52 vars in 829 bytes} [Sun Jul 24 19:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:57:23.543428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ce168-c893-4673-9528-9fd5dc648134 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:23.624741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ce168-c893-4673-9528-9fd5dc648134 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:23.630794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:23.631928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:23.632309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:23.632718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] PATH_INFO: `/v3/users/f95a2d31bda84b15808841a0c261bb59` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:23.634317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f95a2d31bda84b15808841a0c261bb59)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:23.635691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WARNING py.warnings [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 19:57:23.635691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: warnings.warn(msg) Jul 24 19:57:23.635691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: Jul 24 19:57:23.637459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:23.871641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f95a2d31bda84b15808841a0c261bb59'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:23.892555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f95a2d31bda84b15808841a0c261bb59'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:23.893650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f95a2d31bda84b15808841a0c261bb59'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:23.982125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f95a2d31bda84b15808841a0c261bb59'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:24.005404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:57:24.008223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f95a2d31bda84b15808841a0c261bb59'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:24.016441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 755/1507] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 19:57:23 2022] DELETE /identity/v3/users/f95a2d31bda84b15808841a0c261bb59 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:57:24.021910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15111b1f-0a7b-4079-807e-c6550d9786d8 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:24.023403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15111b1f-0a7b-4079-807e-c6550d9786d8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:24.023403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15111b1f-0a7b-4079-807e-c6550d9786d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:24.045077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-15111b1f-0a7b-4079-807e-c6550d9786d8 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:57:24.047669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-15111b1f-0a7b-4079-807e-c6550d9786d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:57:24.062905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:57:24.115762 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 753/1508] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 19:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:57:24.127359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:24.129194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:57:24.222580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0559b-9f57-4a01-8447-ff34a2497412 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:24.226956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6315b-09ed-4308-aca9-395e98920261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:24.227633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6315b-09ed-4308-aca9-395e98920261 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:24.227923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6315b-09ed-4308-aca9-395e98920261 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:24.228199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6315b-09ed-4308-aca9-395e98920261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:24.238851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6315b-09ed-4308-aca9-395e98920261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:24.240422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6315b-09ed-4308-aca9-395e98920261 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:24.254680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 756/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 19:57:24.841809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15111b1f-0a7b-4079-807e-c6550d9786d8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:57:24.843348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-15111b1f-0a7b-4079-807e-c6550d9786d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:57:24.855982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15111b1f-0a7b-4079-807e-c6550d9786d8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:57:24.861569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:57:24.862477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:57:24.862857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:57:24.863040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] PATH_INFO: `/v3/projects/e80d332f409e4f0aac4e3d56193ae95c` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:57:24.864244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e80d332f409e4f0aac4e3d56193ae95c)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:57:24.865308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING py.warnings [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 19:57:24.865308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: warnings.warn(msg) Jul 24 19:57:24.865308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: Jul 24 19:57:24.866505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:57:24.965580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e80d332f409e4f0aac4e3d56193ae95c', 'request_id': 'req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:24.971621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e80d332f409e4f0aac4e3d56193ae95c', 'request_id': 'req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:24.985783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e80d332f409e4f0aac4e3d56193ae95c', 'request_id': 'req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:25.039536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] The token cache is being invalidate because project e80d332f409e4f0aac4e3d56193ae95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 19:57:25.039827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:25.040491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-75f0d48d-4f09-4199-af8b-5d1efb4b1dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 19:57:25.042292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 754/1510] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 19:57:24 2022] DELETE /identity/v3/projects/e80d332f409e4f0aac4e3d56193ae95c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 19:57:34.246159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:57:35.035905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:59:03.358465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb65abd-239e-4b4d-a633-5030b96e5673 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:03.358883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb65abd-239e-4b4d-a633-5030b96e5673 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:03.359028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb65abd-239e-4b4d-a633-5030b96e5673 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:03.360427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:59:03.366017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6093143-3328-4706-ae10-dba49450e9fd None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:03.366555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6093143-3328-4706-ae10-dba49450e9fd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:03.366695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6093143-3328-4706-ae10-dba49450e9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:03.386949 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-b6093143-3328-4706-ae10-dba49450e9fd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:03.390036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b6093143-3328-4706-ae10-dba49450e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:03.453362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:59:03.615859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb65abd-239e-4b4d-a633-5030b96e5673 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:59:03.617659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7bb65abd-239e-4b4d-a633-5030b96e5673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:03.720200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb65abd-239e-4b4d-a633-5030b96e5673 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:59:03.725874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a65b4a-85a4-49c8-9106-da4b56f07d62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:59:03.726536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a65b4a-85a4-49c8-9106-da4b56f07d62 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:03.726824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a65b4a-85a4-49c8-9106-da4b56f07d62 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:03.727103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a65b4a-85a4-49c8-9106-da4b56f07d62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:03.737490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a65b4a-85a4-49c8-9106-da4b56f07d62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:59:03.739126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a65b4a-85a4-49c8-9106-da4b56f07d62 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:59:03.749715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 758/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:59:04.594118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a356c7-0683-4987-aec7-10dba792d438 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:04.598106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a356c7-0683-4987-aec7-10dba792d438 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:04.598106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a356c7-0683-4987-aec7-10dba792d438 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:04.598106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:59:04.602933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707f5aa-b262-4a87-8de8-a0b27384290c None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:04.603461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707f5aa-b262-4a87-8de8-a0b27384290c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:04.603461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707f5aa-b262-4a87-8de8-a0b27384290c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:04.623332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d707f5aa-b262-4a87-8de8-a0b27384290c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:04.624562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d707f5aa-b262-4a87-8de8-a0b27384290c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:04.633761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:59:04.786800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a356c7-0683-4987-aec7-10dba792d438 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:59:04.801640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a356c7-0683-4987-aec7-10dba792d438 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:59:04.807011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3a9f4-aa67-4731-afcc-d12519603bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:59:04.807663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3a9f4-aa67-4731-afcc-d12519603bb4 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:04.807948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3a9f4-aa67-4731-afcc-d12519603bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:04.808229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3a9f4-aa67-4731-afcc-d12519603bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:04.817417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3a9f4-aa67-4731-afcc-d12519603bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:59:04.819017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3a9f4-aa67-4731-afcc-d12519603bb4 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:59:04.830128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 757/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:59:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:59:05.550524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16a515-4571-4961-9a28-f4345ac10f24 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:05.550836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16a515-4571-4961-9a28-f4345ac10f24 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:05.551147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16a515-4571-4961-9a28-f4345ac10f24 None None] PATH_INFO: `/v3` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:05.552680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 760/1517] 10.222.0.42 () {54 vars in 925 bytes} [Sun Jul 24 19:59:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 19:59:05.558714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff321fe-fb56-4d9f-850e-3f86a8755edb None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:05.559034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff321fe-fb56-4d9f-850e-3f86a8755edb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:05.559288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff321fe-fb56-4d9f-850e-3f86a8755edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:05.611729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-aff321fe-fb56-4d9f-850e-3f86a8755edb None None] MFA Rules not processed for user `1c1b23b83544423abc6919d805916b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:05.612878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-aff321fe-fb56-4d9f-850e-3f86a8755edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:05.688732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 758/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Sun Jul 24 19:59:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:59:09.731838 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e857-1a71-411b-8c73-6817bf5f34f4 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:09.732409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e857-1a71-411b-8c73-6817bf5f34f4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:09.732409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e857-1a71-411b-8c73-6817bf5f34f4 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:09.733851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 761/1519] 10.222.0.17 () {54 vars in 961 bytes} [Sun Jul 24 19:59:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:59:09.739256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270aef8-68e2-4e39-83ec-e5f9cf977aac None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:09.739724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270aef8-68e2-4e39-83ec-e5f9cf977aac None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:09.739827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270aef8-68e2-4e39-83ec-e5f9cf977aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:09.758721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-5270aef8-68e2-4e39-83ec-e5f9cf977aac None None] MFA Rules not processed for user `1c1b23b83544423abc6919d805916b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:09.759952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5270aef8-68e2-4e39-83ec-e5f9cf977aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:09.768691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 759/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Sun Jul 24 19:59:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:59:09.776291 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e857-1a71-411b-8c73-6817bf5f34f4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:59:09.789911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e857-1a71-411b-8c73-6817bf5f34f4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:59:09.795428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa7754-f7e0-4235-8ce7-59c6d3901909 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:59:09.796075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa7754-f7e0-4235-8ce7-59c6d3901909 service placement] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:09.796360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa7754-f7e0-4235-8ce7-59c6d3901909 service placement] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:09.796663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa7754-f7e0-4235-8ce7-59c6d3901909 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:09.805300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa7754-f7e0-4235-8ce7-59c6d3901909 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:59:09.806881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa7754-f7e0-4235-8ce7-59c6d3901909 service placement] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:59:09.817671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 762/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:59:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:59:16.133037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf45376-4c8f-40c9-a117-cf49a58b482b None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:16.133687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf45376-4c8f-40c9-a117-cf49a58b482b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:16.133687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf45376-4c8f-40c9-a117-cf49a58b482b None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:16.135025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:59:16.140375 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382f28-f2ad-4339-b771-904f0b3356fa None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:16.141017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382f28-f2ad-4339-b771-904f0b3356fa None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:16.141017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382f28-f2ad-4339-b771-904f0b3356fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:16.160187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-72382f28-f2ad-4339-b771-904f0b3356fa None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:16.161440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-72382f28-f2ad-4339-b771-904f0b3356fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:16.170286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:59:16.330623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf45376-4c8f-40c9-a117-cf49a58b482b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:59:16.345451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf45376-4c8f-40c9-a117-cf49a58b482b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:59:16.351017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a87973-98de-4889-a79f-1293e4c73f8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:59:16.351668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a87973-98de-4889-a79f-1293e4c73f8d service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:16.351950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a87973-98de-4889-a79f-1293e4c73f8d service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:16.352225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a87973-98de-4889-a79f-1293e4c73f8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:16.362278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a87973-98de-4889-a79f-1293e4c73f8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:59:16.363882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a87973-98de-4889-a79f-1293e4c73f8d service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:59:16.374007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 761/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:59:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:59:17.201878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944de86d-90b6-4996-8dec-9fe5b5f0ffc4 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:17.202574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944de86d-90b6-4996-8dec-9fe5b5f0ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:17.202574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944de86d-90b6-4996-8dec-9fe5b5f0ffc4 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:17.203985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 24 19:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:59:17.209070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c64d0b-5f7a-4623-82ad-6bf514e48b76 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:17.209554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c64d0b-5f7a-4623-82ad-6bf514e48b76 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:17.209668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c64d0b-5f7a-4623-82ad-6bf514e48b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:17.227669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-66c64d0b-5f7a-4623-82ad-6bf514e48b76 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:17.228889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-66c64d0b-5f7a-4623-82ad-6bf514e48b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:17.237427 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 24 19:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:59:17.392547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944de86d-90b6-4996-8dec-9fe5b5f0ffc4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:59:17.407566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944de86d-90b6-4996-8dec-9fe5b5f0ffc4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:59:17.413060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f975d7-8d57-44af-8d45-6bda7ef6f725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:59:17.413718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f975d7-8d57-44af-8d45-6bda7ef6f725 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:17.414007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f975d7-8d57-44af-8d45-6bda7ef6f725 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:17.414288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f975d7-8d57-44af-8d45-6bda7ef6f725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:17.424507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f975d7-8d57-44af-8d45-6bda7ef6f725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:59:17.426146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f975d7-8d57-44af-8d45-6bda7ef6f725 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:59:17.436686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 765/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:59:21.238418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca74110-260c-47d3-8665-b5cbeded4772 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:21.239015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca74110-260c-47d3-8665-b5cbeded4772 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:21.239123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca74110-260c-47d3-8665-b5cbeded4772 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:21.240458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 24 19:59:21.246976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583d6cd-34a5-4105-860c-507ec486109e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:21.247940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583d6cd-34a5-4105-860c-507ec486109e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:21.248104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583d6cd-34a5-4105-860c-507ec486109e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:21.266555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d583d6cd-34a5-4105-860c-507ec486109e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:21.267773 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d583d6cd-34a5-4105-860c-507ec486109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:21.275684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:59:21.359277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca74110-260c-47d3-8665-b5cbeded4772 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:59:21.361170 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8ca74110-260c-47d3-8665-b5cbeded4772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:21.390762 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:59:21.421765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca74110-260c-47d3-8665-b5cbeded4772 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:59:21.425661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be932293-912e-407e-8141-b5426bfb4663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:59:21.426318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be932293-912e-407e-8141-b5426bfb4663 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:21.426601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be932293-912e-407e-8141-b5426bfb4663 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:21.427239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be932293-912e-407e-8141-b5426bfb4663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:21.436817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be932293-912e-407e-8141-b5426bfb4663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:59:21.438237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be932293-912e-407e-8141-b5426bfb4663 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:59:21.447481 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 764/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:59:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:59:23.496076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f823ff-3a48-4338-8fa7-c20f745f6eed None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:23.496631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f823ff-3a48-4338-8fa7-c20f745f6eed None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:23.496631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f823ff-3a48-4338-8fa7-c20f745f6eed None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:23.498075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:59:23.503517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b084f39-adc8-44fa-892d-6c1cd124090e None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:23.503959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b084f39-adc8-44fa-892d-6c1cd124090e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:23.504088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b084f39-adc8-44fa-892d-6c1cd124090e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:23.522656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1b084f39-adc8-44fa-892d-6c1cd124090e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:23.523869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1b084f39-adc8-44fa-892d-6c1cd124090e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:23.532746 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 19:59:23.570080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f823ff-3a48-4338-8fa7-c20f745f6eed None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:59:23.581307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f823ff-3a48-4338-8fa7-c20f745f6eed None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:59:23.585034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8354a-000b-40cc-95df-70240d4e2bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:59:23.585681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8354a-000b-40cc-95df-70240d4e2bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:23.585965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8354a-000b-40cc-95df-70240d4e2bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:23.586241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8354a-000b-40cc-95df-70240d4e2bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:23.596099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae8354a-000b-40cc-95df-70240d4e2bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:59:23.597576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae8354a-000b-40cc-95df-70240d4e2bd0 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:59:23.607985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 768/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:59:27.074677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a2b89-70af-43f3-b877-b983257d158b None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:27.075788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a2b89-70af-43f3-b877-b983257d158b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:27.075788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a2b89-70af-43f3-b877-b983257d158b None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:27.077175 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:59:27.082334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5d85a-5c47-4e54-917a-20b01c2ed936 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:27.082749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5d85a-5c47-4e54-917a-20b01c2ed936 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:27.082894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5d85a-5c47-4e54-917a-20b01c2ed936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:27.089137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=74 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:59:27.102620 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a2c5d85a-5c47-4e54-917a-20b01c2ed936 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:27.103847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a2c5d85a-5c47-4e54-917a-20b01c2ed936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:27.112046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:59:27.421744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a2b89-70af-43f3-b877-b983257d158b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:59:27.423715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-837a2b89-70af-43f3-b877-b983257d158b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:27.480716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a2b89-70af-43f3-b877-b983257d158b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:59:27.484244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a3792-7890-45aa-b541-9d99a3b0d054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:59:27.484921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a3792-7890-45aa-b541-9d99a3b0d054 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:27.485214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a3792-7890-45aa-b541-9d99a3b0d054 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:27.485496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a3792-7890-45aa-b541-9d99a3b0d054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:27.494558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a3792-7890-45aa-b541-9d99a3b0d054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:59:27.496004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a3792-7890-45aa-b541-9d99a3b0d054 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:59:27.506141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 767/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:59:29.591288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0e169-e20d-45be-8060-bb076be39d93 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:29.591993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0e169-e20d-45be-8060-bb076be39d93 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:29.591993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0e169-e20d-45be-8060-bb076be39d93 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:29.593282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 24 19:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 19:59:29.598897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808837d-5f4b-4c27-97b9-1d56d0b879f2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:29.599305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808837d-5f4b-4c27-97b9-1d56d0b879f2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:29.599463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808837d-5f4b-4c27-97b9-1d56d0b879f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:29.617985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d808837d-5f4b-4c27-97b9-1d56d0b879f2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 19:59:29.619193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d808837d-5f4b-4c27-97b9-1d56d0b879f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:29.627245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 24 19:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 19:59:29.940709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0e169-e20d-45be-8060-bb076be39d93 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 19:59:29.942451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9ca0e169-e20d-45be-8060-bb076be39d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 19:59:29.956340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0e169-e20d-45be-8060-bb076be39d93 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 19:59:29.960033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbccb62-75a7-4295-8b5e-4e963d953b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 19:59:29.960702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbccb62-75a7-4295-8b5e-4e963d953b33 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 19:59:29.960991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbccb62-75a7-4295-8b5e-4e963d953b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 19:59:29.961270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbccb62-75a7-4295-8b5e-4e963d953b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 19:59:29.971268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbccb62-75a7-4295-8b5e-4e963d953b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 19:59:29.972734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbccb62-75a7-4295-8b5e-4e963d953b33 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 19:59:29.982346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 771/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 19:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 19:59:39.612716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=47 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 19:59:39.977955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=8 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:00:57.936953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa098e-8260-403c-b76d-f3323838cfef None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:57.937595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa098e-8260-403c-b76d-f3323838cfef None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:57.937595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa098e-8260-403c-b76d-f3323838cfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:57.965463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-81fa098e-8260-403c-b76d-f3323838cfef None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:57.966879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-81fa098e-8260-403c-b76d-f3323838cfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:57.981651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 769/1540] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:57.991317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900e694-2e3b-434e-956c-d522d413f198 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:57.992409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900e694-2e3b-434e-956c-d522d413f198 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:57.993503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900e694-2e3b-434e-956c-d522d413f198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.031045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c900e694-2e3b-434e-956c-d522d413f198 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.032759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c900e694-2e3b-434e-956c-d522d413f198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.050363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 772/1541] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.055915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa098e-8260-403c-b76d-f3323838cfef None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.068829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa098e-8260-403c-b76d-f3323838cfef None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.077523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3996d-e127-4939-b62e-0e55235bdb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.078214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3996d-e127-4939-b62e-0e55235bdb80 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.078506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3996d-e127-4939-b62e-0e55235bdb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.078783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3996d-e127-4939-b62e-0e55235bdb80 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.079533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3996d-e127-4939-b62e-0e55235bdb80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:00:58.079841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3996d-e127-4939-b62e-0e55235bdb80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.081518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3996d-e127-4939-b62e-0e55235bdb80 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.107280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 770/1542] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:00:58.115946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900e694-2e3b-434e-956c-d522d413f198 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.130367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900e694-2e3b-434e-956c-d522d413f198 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.137073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecb959-19ee-4adc-8209-a0ad18fc598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.138090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecb959-19ee-4adc-8209-a0ad18fc598b admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.138688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecb959-19ee-4adc-8209-a0ad18fc598b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.138832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785e9ca-03a7-46bd-900f-56c62597bc45 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.139031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785e9ca-03a7-46bd-900f-56c62597bc45 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.139307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785e9ca-03a7-46bd-900f-56c62597bc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.139847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecb959-19ee-4adc-8209-a0ad18fc598b admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.140836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecb959-19ee-4adc-8209-a0ad18fc598b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.143031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecb959-19ee-4adc-8209-a0ad18fc598b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.169900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-b785e9ca-03a7-46bd-900f-56c62597bc45 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.172315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b785e9ca-03a7-46bd-900f-56c62597bc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.189574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 771/1543] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.203933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a459c-c924-4a13-a5cf-88966e0de1cd None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.204281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a459c-c924-4a13-a5cf-88966e0de1cd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.204508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a459c-c924-4a13-a5cf-88966e0de1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.242003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ae9a459c-c924-4a13-a5cf-88966e0de1cd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.243280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ae9a459c-c924-4a13-a5cf-88966e0de1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.252135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 772/1544] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:00:58.258553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a40cf-5516-4f03-8d37-34fd2e823ca5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.258553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a40cf-5516-4f03-8d37-34fd2e823ca5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.258553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a40cf-5516-4f03-8d37-34fd2e823ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.287407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a65a40cf-5516-4f03-8d37-34fd2e823ca5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.287407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a65a40cf-5516-4f03-8d37-34fd2e823ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.290232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 773/1545] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:58.291890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e252f60-4e7c-4992-a81c-a1b2da98f8ee None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.292548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e252f60-4e7c-4992-a81c-a1b2da98f8ee None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.296720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e252f60-4e7c-4992-a81c-a1b2da98f8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.327796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6e252f60-4e7c-4992-a81c-a1b2da98f8ee None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.329533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6e252f60-4e7c-4992-a81c-a1b2da98f8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.440957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 773/1546] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.442202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a40cf-5516-4f03-8d37-34fd2e823ca5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.454593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a40cf-5516-4f03-8d37-34fd2e823ca5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.457998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 774/1547] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.458880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e252f60-4e7c-4992-a81c-a1b2da98f8ee None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.466919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1b435-25d5-424c-8059-61d714e5b0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.467756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1b435-25d5-424c-8059-61d714e5b0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.468071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1b435-25d5-424c-8059-61d714e5b0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.468202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1b435-25d5-424c-8059-61d714e5b0b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.468973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1b435-25d5-424c-8059-61d714e5b0b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:00:58.469287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1b435-25d5-424c-8059-61d714e5b0b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.470767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e252f60-4e7c-4992-a81c-a1b2da98f8ee None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.470915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1b435-25d5-424c-8059-61d714e5b0b9 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.476632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba15ec-fac5-4c7c-8c29-041a343bdefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.478204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba15ec-fac5-4c7c-8c29-041a343bdefb admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.478550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba15ec-fac5-4c7c-8c29-041a343bdefb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.478912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba15ec-fac5-4c7c-8c29-041a343bdefb admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.479677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba15ec-fac5-4c7c-8c29-041a343bdefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.480819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WARNING py.warnings [None req-ffba15ec-fac5-4c7c-8c29-041a343bdefb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 20:00:58.480819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: warnings.warn(msg) Jul 24 20:00:58.480819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: Jul 24 20:00:58.482133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba15ec-fac5-4c7c-8c29-041a343bdefb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.484755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 774/1548] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:00:58.486222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a66704-b200-40cf-b6bb-f5258ee7fc2f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.486520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a66704-b200-40cf-b6bb-f5258ee7fc2f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.486804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a66704-b200-40cf-b6bb-f5258ee7fc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.512853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-b3a66704-b200-40cf-b6bb-f5258ee7fc2f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.514021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b3a66704-b200-40cf-b6bb-f5258ee7fc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.522210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 775/1549] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.523702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad27ebd-ec81-4473-b79e-56acb6d1eddf None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.524039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad27ebd-ec81-4473-b79e-56acb6d1eddf None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.524405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad27ebd-ec81-4473-b79e-56acb6d1eddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.545220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-4ad27ebd-ec81-4473-b79e-56acb6d1eddf None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.546721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4ad27ebd-ec81-4473-b79e-56acb6d1eddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.554794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 776/1550] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.562429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1419158-cbd0-49b3-a2d0-dcb7b1bcb8a4 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.562429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1419158-cbd0-49b3-a2d0-dcb7b1bcb8a4 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.562429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1419158-cbd0-49b3-a2d0-dcb7b1bcb8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.568365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 775/1551] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:58.573726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aba15f-b948-46d4-a27e-dfd56aaf00a6 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.573726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aba15f-b948-46d4-a27e-dfd56aaf00a6 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.573726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aba15f-b948-46d4-a27e-dfd56aaf00a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.581183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e1419158-cbd0-49b3-a2d0-dcb7b1bcb8a4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.582408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e1419158-cbd0-49b3-a2d0-dcb7b1bcb8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.588916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d2aba15f-b948-46d4-a27e-dfd56aaf00a6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.590252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d2aba15f-b948-46d4-a27e-dfd56aaf00a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.590681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 777/1552] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.591446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1419158-cbd0-49b3-a2d0-dcb7b1bcb8a4 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.599994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 776/1553] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.602696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76ea9d-a735-47a2-af2e-cb19e4a265a9 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.602809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1419158-cbd0-49b3-a2d0-dcb7b1bcb8a4 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.602908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76ea9d-a735-47a2-af2e-cb19e4a265a9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.602908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76ea9d-a735-47a2-af2e-cb19e4a265a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.608037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4888c03-e88e-41aa-94d5-2e476f789867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.608740 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4888c03-e88e-41aa-94d5-2e476f789867 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.609091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4888c03-e88e-41aa-94d5-2e476f789867 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.609381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4888c03-e88e-41aa-94d5-2e476f789867 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.610084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4888c03-e88e-41aa-94d5-2e476f789867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.611130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING py.warnings [None req-b4888c03-e88e-41aa-94d5-2e476f789867 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 20:00:58.611130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: warnings.warn(msg) Jul 24 20:00:58.611130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: Jul 24 20:00:58.612870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4888c03-e88e-41aa-94d5-2e476f789867 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.623150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-0f76ea9d-a735-47a2-af2e-cb19e4a265a9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.624374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0f76ea9d-a735-47a2-af2e-cb19e4a265a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.632279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 777/1554] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.633773 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc7fad-1509-43b4-bb40-780f678bc791 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.634097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc7fad-1509-43b4-bb40-780f678bc791 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.634486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc7fad-1509-43b4-bb40-780f678bc791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.654397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-15cc7fad-1509-43b4-bb40-780f678bc791 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.655609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-15cc7fad-1509-43b4-bb40-780f678bc791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.663613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 778/1555] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.665499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1960-1e3e-4280-9783-a7628bce2700 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.665789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1960-1e3e-4280-9783-a7628bce2700 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.666070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1960-1e3e-4280-9783-a7628bce2700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.730488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 778/1556] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/projects => generated 389 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:58.730488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ec034-130a-4fbc-af5d-e78d83284090 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.730488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ec034-130a-4fbc-af5d-e78d83284090 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.730488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ec034-130a-4fbc-af5d-e78d83284090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.749607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f90ec034-130a-4fbc-af5d-e78d83284090 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.750859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f90ec034-130a-4fbc-af5d-e78d83284090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.817969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 779/1557] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.818876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ec034-130a-4fbc-af5d-e78d83284090 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.822572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-1fbb1960-1e3e-4280-9783-a7628bce2700 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.823899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1fbb1960-1e3e-4280-9783-a7628bce2700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.828668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ec034-130a-4fbc-af5d-e78d83284090 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.832688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 779/1558] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.833496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb1960-1e3e-4280-9783-a7628bce2700 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.837564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f0d05-9964-47dc-a9d5-c8fa07afa185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.838251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f0d05-9964-47dc-a9d5-c8fa07afa185 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.838542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f0d05-9964-47dc-a9d5-c8fa07afa185 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.838819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f0d05-9964-47dc-a9d5-c8fa07afa185 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.839563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f0d05-9964-47dc-a9d5-c8fa07afa185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.841240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f0d05-9964-47dc-a9d5-c8fa07afa185 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.843971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb1960-1e3e-4280-9783-a7628bce2700 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.849465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5730a-f1c2-44d0-bb44-782f842270b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.850143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5730a-f1c2-44d0-bb44-782f842270b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.850431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5730a-f1c2-44d0-bb44-782f842270b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.850712 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5730a-f1c2-44d0-bb44-782f842270b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.851446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5730a-f1c2-44d0-bb44-782f842270b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:00:58.851757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5730a-f1c2-44d0-bb44-782f842270b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.856048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 780/1559] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:00:58.856189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WARNING py.warnings [None req-cff5730a-f1c2-44d0-bb44-782f842270b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 20:00:58.856189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: warnings.warn(msg) Jul 24 20:00:58.856189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: Jul 24 20:00:58.856390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f0d05-9964-47dc-a9d5-c8fa07afa185 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.856483 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5730a-f1c2-44d0-bb44-782f842270b6 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.858937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:00:58.864109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f0d05-9964-47dc-a9d5-c8fa07afa185 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.867235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 780/1560] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:00:58.868836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae65648-4c5d-42a8-afca-1f4c29530a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.869090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612be533-d956-475e-bdd3-fab8d952e480 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.869382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612be533-d956-475e-bdd3-fab8d952e480 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.869499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae65648-4c5d-42a8-afca-1f4c29530a55 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.869690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612be533-d956-475e-bdd3-fab8d952e480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.869800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae65648-4c5d-42a8-afca-1f4c29530a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.870040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae65648-4c5d-42a8-afca-1f4c29530a55 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.870774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae65648-4c5d-42a8-afca-1f4c29530a55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:00:58.871081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae65648-4c5d-42a8-afca-1f4c29530a55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.872689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae65648-4c5d-42a8-afca-1f4c29530a55 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.885677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 781/1561] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:00:58.886508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae65648-4c5d-42a8-afca-1f4c29530a55 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.888353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-612be533-d956-475e-bdd3-fab8d952e480 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.889590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-612be533-d956-475e-bdd3-fab8d952e480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.896702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae65648-4c5d-42a8-afca-1f4c29530a55 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.898523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 781/1562] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.899845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612be533-d956-475e-bdd3-fab8d952e480 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.902628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e157e7-f53c-4439-935d-aa139c50c730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.903279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e157e7-f53c-4439-935d-aa139c50c730 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.903565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e157e7-f53c-4439-935d-aa139c50c730 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.903847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e157e7-f53c-4439-935d-aa139c50c730 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.904621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e157e7-f53c-4439-935d-aa139c50c730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:00:58.905079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e157e7-f53c-4439-935d-aa139c50c730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.906633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e157e7-f53c-4439-935d-aa139c50c730 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.909521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612be533-d956-475e-bdd3-fab8d952e480 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.914336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461d1a7-3f4d-4b31-a6ab-d8c1121a11ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.915742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461d1a7-3f4d-4b31-a6ab-d8c1121a11ac admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.916026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461d1a7-3f4d-4b31-a6ab-d8c1121a11ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.916303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461d1a7-3f4d-4b31-a6ab-d8c1121a11ac admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.917027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461d1a7-3f4d-4b31-a6ab-d8c1121a11ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.918938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461d1a7-3f4d-4b31-a6ab-d8c1121a11ac admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.919666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 782/1563] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:00:58.921236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4262da-1f84-4133-896c-d7e56f9c5e60 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.921539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4262da-1f84-4133-896c-d7e56f9c5e60 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.921818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4262da-1f84-4133-896c-d7e56f9c5e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.939625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-fe4262da-1f84-4133-896c-d7e56f9c5e60 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:00:58.940859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fe4262da-1f84-4133-896c-d7e56f9c5e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:00:58.948868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 783/1564] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:00:58.949658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4262da-1f84-4133-896c-d7e56f9c5e60 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.958902 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4262da-1f84-4133-896c-d7e56f9c5e60 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.963964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a134c-674e-4a37-ba64-640cad6db80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.964607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a134c-674e-4a37-ba64-640cad6db80f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.964916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a134c-674e-4a37-ba64-640cad6db80f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.965195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a134c-674e-4a37-ba64-640cad6db80f admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.965916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a134c-674e-4a37-ba64-640cad6db80f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:00:58.966222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a134c-674e-4a37-ba64-640cad6db80f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:58.967806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a134c-674e-4a37-ba64-640cad6db80f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:58.979918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 784/1565] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:00:58.984812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a134c-674e-4a37-ba64-640cad6db80f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:58.991609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a134c-674e-4a37-ba64-640cad6db80f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:58.996481 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adbb51-21a2-488f-8f7e-0ea8dd2141c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:58.997250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adbb51-21a2-488f-8f7e-0ea8dd2141c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:58.997541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adbb51-21a2-488f-8f7e-0ea8dd2141c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:58.997824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adbb51-21a2-488f-8f7e-0ea8dd2141c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:58.998770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adbb51-21a2-488f-8f7e-0ea8dd2141c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.000132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adbb51-21a2-488f-8f7e-0ea8dd2141c6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.010606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 782/1566] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:00:58 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:59.010606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461d1a7-3f4d-4b31-a6ab-d8c1121a11ac admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.015540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 785/1567] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:00:59.015540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adbb51-21a2-488f-8f7e-0ea8dd2141c6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.020978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461d1a7-3f4d-4b31-a6ab-d8c1121a11ac admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.021600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adbb51-21a2-488f-8f7e-0ea8dd2141c6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.026118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d831a7d-3a68-4a1f-a2f5-6366d2e8300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.026782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d831a7d-3a68-4a1f-a2f5-6366d2e8300f admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.027068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d831a7d-3a68-4a1f-a2f5-6366d2e8300f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.027331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5deec-2e7c-49b9-bca0-6bfb96191ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.027459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d831a7d-3a68-4a1f-a2f5-6366d2e8300f admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.027980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5deec-2e7c-49b9-bca0-6bfb96191ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.028096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d831a7d-3a68-4a1f-a2f5-6366d2e8300f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.028274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5deec-2e7c-49b9-bca0-6bfb96191ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.028541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5deec-2e7c-49b9-bca0-6bfb96191ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.029294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5deec-2e7c-49b9-bca0-6bfb96191ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.029940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d831a7d-3a68-4a1f-a2f5-6366d2e8300f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.031292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5deec-2e7c-49b9-bca0-6bfb96191ee0 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.127036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 783/1568] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:59.127954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d831a7d-3a68-4a1f-a2f5-6366d2e8300f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.159076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:00:59.163974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 786/1569] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:59.163974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5deec-2e7c-49b9-bca0-6bfb96191ee0 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.190570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d831a7d-3a68-4a1f-a2f5-6366d2e8300f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.225910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5deec-2e7c-49b9-bca0-6bfb96191ee0 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.230716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf0a5f-8cb8-473b-876b-2a7f0d340777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.231389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf0a5f-8cb8-473b-876b-2a7f0d340777 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.231979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf0a5f-8cb8-473b-876b-2a7f0d340777 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.231979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf0a5f-8cb8-473b-876b-2a7f0d340777 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.232731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf0a5f-8cb8-473b-876b-2a7f0d340777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.234669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf0a5f-8cb8-473b-876b-2a7f0d340777 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.250875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd3e08-81aa-4cec-b46b-f4efe994b564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.251590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd3e08-81aa-4cec-b46b-f4efe994b564 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.251876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd3e08-81aa-4cec-b46b-f4efe994b564 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.252154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd3e08-81aa-4cec-b46b-f4efe994b564 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.252912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd3e08-81aa-4cec-b46b-f4efe994b564 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:00:59.253223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd3e08-81aa-4cec-b46b-f4efe994b564 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.254845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd3e08-81aa-4cec-b46b-f4efe994b564 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.267300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 784/1570] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:00:59.268125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd3e08-81aa-4cec-b46b-f4efe994b564 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.278716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd3e08-81aa-4cec-b46b-f4efe994b564 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.283376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36a84b-e2d5-4784-a590-61ea6fc4a6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.284031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36a84b-e2d5-4784-a590-61ea6fc4a6bd admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.284312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36a84b-e2d5-4784-a590-61ea6fc4a6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.284590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36a84b-e2d5-4784-a590-61ea6fc4a6bd admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.285343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36a84b-e2d5-4784-a590-61ea6fc4a6bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:00:59.285646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36a84b-e2d5-4784-a590-61ea6fc4a6bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.287429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36a84b-e2d5-4784-a590-61ea6fc4a6bd admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.299742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 785/1571] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:00:59.300580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36a84b-e2d5-4784-a590-61ea6fc4a6bd admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.310228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36a84b-e2d5-4784-a590-61ea6fc4a6bd admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.315093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d875aa-0615-4793-b08f-d38354e65d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.319786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d875aa-0615-4793-b08f-d38354e65d49 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.319897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 787/1572] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/projects => generated 357 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:59.319995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d875aa-0615-4793-b08f-d38354e65d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.319995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d875aa-0615-4793-b08f-d38354e65d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.320157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf0a5f-8cb8-473b-876b-2a7f0d340777 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.320258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d875aa-0615-4793-b08f-d38354e65d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.320258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d875aa-0615-4793-b08f-d38354e65d49 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.379678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf0a5f-8cb8-473b-876b-2a7f0d340777 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.385441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ddea9-c657-4eee-9fc3-8c9c1ef08a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.386115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ddea9-c657-4eee-9fc3-8c9c1ef08a11 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.386401 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ddea9-c657-4eee-9fc3-8c9c1ef08a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.386679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ddea9-c657-4eee-9fc3-8c9c1ef08a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.387407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ddea9-c657-4eee-9fc3-8c9c1ef08a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.389057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ddea9-c657-4eee-9fc3-8c9c1ef08a11 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.403783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 788/1573] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:00:59.403783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ddea9-c657-4eee-9fc3-8c9c1ef08a11 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.404107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 786/1574] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:59.404107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d875aa-0615-4793-b08f-d38354e65d49 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.411801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:00:59.416934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ddea9-c657-4eee-9fc3-8c9c1ef08a11 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.469658 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d875aa-0615-4793-b08f-d38354e65d49 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.474690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44773d3-af94-4676-8401-97d6bb91291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.475381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44773d3-af94-4676-8401-97d6bb91291b admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.475670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44773d3-af94-4676-8401-97d6bb91291b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.475949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44773d3-af94-4676-8401-97d6bb91291b admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.476661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44773d3-af94-4676-8401-97d6bb91291b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.478056 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42399e64-883d-462c-bb26-81dfd62fe186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.478708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44773d3-af94-4676-8401-97d6bb91291b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.478962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42399e64-883d-462c-bb26-81dfd62fe186 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.479251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42399e64-883d-462c-bb26-81dfd62fe186 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.479530 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42399e64-883d-462c-bb26-81dfd62fe186 admin admin] PATH_INFO: `/v3/projects/54d28b408f5e445f98ae90382124f24d/users/793472a527614ba0905c0411d56056f0/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.494219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42399e64-883d-462c-bb26-81dfd62fe186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54d28b408f5e445f98ae90382124f24d, user_id=793472a527614ba0905c0411d56056f0, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.495285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING py.warnings [None req-42399e64-883d-462c-bb26-81dfd62fe186 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 20:00:59.495285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: warnings.warn(msg) Jul 24 20:00:59.495285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: Jul 24 20:00:59.497223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42399e64-883d-462c-bb26-81dfd62fe186 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.577337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 787/1575] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:59.578354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44773d3-af94-4676-8401-97d6bb91291b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.589319 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44773d3-af94-4676-8401-97d6bb91291b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.594382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca4e3-7dbc-49d1-b893-f79b249374f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.595101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca4e3-7dbc-49d1-b893-f79b249374f4 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.595391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca4e3-7dbc-49d1-b893-f79b249374f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.595672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca4e3-7dbc-49d1-b893-f79b249374f4 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.596388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dca4e3-7dbc-49d1-b893-f79b249374f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.598490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dca4e3-7dbc-49d1-b893-f79b249374f4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.622952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 789/1576] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:00:59 2022] PUT /identity/v3/projects/54d28b408f5e445f98ae90382124f24d/users/793472a527614ba0905c0411d56056f0/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:00:59.632264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42399e64-883d-462c-bb26-81dfd62fe186 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.693059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42399e64-883d-462c-bb26-81dfd62fe186 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.698564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d9371-2afe-4692-ae4d-82f9b6b12cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.699239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d9371-2afe-4692-ae4d-82f9b6b12cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.699528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d9371-2afe-4692-ae4d-82f9b6b12cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.699809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d9371-2afe-4692-ae4d-82f9b6b12cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.700516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d9371-2afe-4692-ae4d-82f9b6b12cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.702529 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d9371-2afe-4692-ae4d-82f9b6b12cb5 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.708829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 788/1577] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:59.709662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca4e3-7dbc-49d1-b893-f79b249374f4 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.719932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca4e3-7dbc-49d1-b893-f79b249374f4 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.724901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6bb83-da73-4574-9168-f8bc16d9d8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.725574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6bb83-da73-4574-9168-f8bc16d9d8ba admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.725861 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6bb83-da73-4574-9168-f8bc16d9d8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.726143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6bb83-da73-4574-9168-f8bc16d9d8ba admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.726841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6bb83-da73-4574-9168-f8bc16d9d8ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.728859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6bb83-da73-4574-9168-f8bc16d9d8ba admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.734576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:00:59.801400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 790/1578] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:59.802232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d9371-2afe-4692-ae4d-82f9b6b12cb5 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.830898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 789/1579] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/projects => generated 393 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:00:59.832111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6bb83-da73-4574-9168-f8bc16d9d8ba admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.865925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d9371-2afe-4692-ae4d-82f9b6b12cb5 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.883512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:00:59.895077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6bb83-da73-4574-9168-f8bc16d9d8ba admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.899843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1596f-a5c5-4077-ac47-1d6ccad628a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.900585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1596f-a5c5-4077-ac47-1d6ccad628a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.900840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1596f-a5c5-4077-ac47-1d6ccad628a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.901119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1596f-a5c5-4077-ac47-1d6ccad628a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.901881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1596f-a5c5-4077-ac47-1d6ccad628a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.903446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1596f-a5c5-4077-ac47-1d6ccad628a8 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.913522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 790/1580] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:00:59.914433 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1596f-a5c5-4077-ac47-1d6ccad628a8 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:00:59.924937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1596f-a5c5-4077-ac47-1d6ccad628a8 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:00:59.926218 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b785b6-dc1c-41e0-b9a4-74a1f01b1fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.926915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b785b6-dc1c-41e0-b9a4-74a1f01b1fea admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.927202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b785b6-dc1c-41e0-b9a4-74a1f01b1fea admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.927483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b785b6-dc1c-41e0-b9a4-74a1f01b1fea admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.928179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b785b6-dc1c-41e0-b9a4-74a1f01b1fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.930110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5f7bd-2b3e-4dd3-9e7c-f35aff75ef42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:00:59.931060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b785b6-dc1c-41e0-b9a4-74a1f01b1fea admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:00:59.931594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5f7bd-2b3e-4dd3-9e7c-f35aff75ef42 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:00:59.931949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5f7bd-2b3e-4dd3-9e7c-f35aff75ef42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:00:59.932243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5f7bd-2b3e-4dd3-9e7c-f35aff75ef42 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:00:59.936265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5f7bd-2b3e-4dd3-9e7c-f35aff75ef42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:00:59.939379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5f7bd-2b3e-4dd3-9e7c-f35aff75ef42 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.005836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 791/1581] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/users => generated 279 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:00.006697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b785b6-dc1c-41e0-b9a4-74a1f01b1fea admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.019603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 791/1582] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:00:59 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:00.020158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b785b6-dc1c-41e0-b9a4-74a1f01b1fea admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.020269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f097f1-e439-4e12-956c-aca389e0bba4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.020269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f097f1-e439-4e12-956c-aca389e0bba4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.020269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f097f1-e439-4e12-956c-aca389e0bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.021868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0d98d-08ab-4a51-acec-68c937b9aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.022541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0d98d-08ab-4a51-acec-68c937b9aae5 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.022831 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0d98d-08ab-4a51-acec-68c937b9aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.023111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0d98d-08ab-4a51-acec-68c937b9aae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.023848 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0d98d-08ab-4a51-acec-68c937b9aae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.025497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0d98d-08ab-4a51-acec-68c937b9aae5 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.039266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 792/1583] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.039266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0d98d-08ab-4a51-acec-68c937b9aae5 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.047953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0d98d-08ab-4a51-acec-68c937b9aae5 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.048451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-36f097f1-e439-4e12-956c-aca389e0bba4 None None] MFA Rules not processed for user `793472a527614ba0905c0411d56056f0`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:00.049862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-36f097f1-e439-4e12-956c-aca389e0bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:00.052560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2094-cdaa-41f3-84b0-02b25a8c1a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.053257 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2094-cdaa-41f3-84b0-02b25a8c1a26 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.053547 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2094-cdaa-41f3-84b0-02b25a8c1a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.053841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2094-cdaa-41f3-84b0-02b25a8c1a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.054637 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d2094-cdaa-41f3-84b0-02b25a8c1a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.056934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d2094-cdaa-41f3-84b0-02b25a8c1a26 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.069393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 793/1584] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.069393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2094-cdaa-41f3-84b0-02b25a8c1a26 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.078906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2094-cdaa-41f3-84b0-02b25a8c1a26 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.083812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd568965-dfb5-409d-b9b3-401500b84ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.084484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd568965-dfb5-409d-b9b3-401500b84ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.084804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd568965-dfb5-409d-b9b3-401500b84ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.085087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd568965-dfb5-409d-b9b3-401500b84ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.085784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd568965-dfb5-409d-b9b3-401500b84ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.087781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd568965-dfb5-409d-b9b3-401500b84ee4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.128686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 792/1585] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:00.128686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f097f1-e439-4e12-956c-aca389e0bba4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.140138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f097f1-e439-4e12-956c-aca389e0bba4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.144666 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62447-0240-48eb-8244-27891346c04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.145321 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62447-0240-48eb-8244-27891346c04d admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.145603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62447-0240-48eb-8244-27891346c04d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.145877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62447-0240-48eb-8244-27891346c04d admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.146563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce62447-0240-48eb-8244-27891346c04d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.148461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce62447-0240-48eb-8244-27891346c04d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.174761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 794/1586] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:00 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:00.175580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd568965-dfb5-409d-b9b3-401500b84ee4 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.186329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd568965-dfb5-409d-b9b3-401500b84ee4 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.191132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3112b4-1c98-47e0-9698-b80b9e0d34d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.192017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3112b4-1c98-47e0-9698-b80b9e0d34d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.192308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3112b4-1c98-47e0-9698-b80b9e0d34d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.192587 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3112b4-1c98-47e0-9698-b80b9e0d34d4 admin admin] PATH_INFO: `/v3/projects/49f6a7894899436d8ce94d9a7500533a/users/55ed193c93c446a8b373f770512e5160/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.207909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3112b4-1c98-47e0-9698-b80b9e0d34d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f6a7894899436d8ce94d9a7500533a, user_id=55ed193c93c446a8b373f770512e5160, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.210627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3112b4-1c98-47e0-9698-b80b9e0d34d4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.254372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 793/1587] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:00 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:00.255376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62447-0240-48eb-8244-27891346c04d admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.265139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62447-0240-48eb-8244-27891346c04d admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.269684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacac71-321f-42d3-9a8f-7239b577a4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.275949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacac71-321f-42d3-9a8f-7239b577a4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.275949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacac71-321f-42d3-9a8f-7239b577a4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.275949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacac71-321f-42d3-9a8f-7239b577a4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.275949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacac71-321f-42d3-9a8f-7239b577a4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.275949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacac71-321f-42d3-9a8f-7239b577a4d3 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.278711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 795/1588] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:00 2022] PUT /identity/v3/projects/49f6a7894899436d8ce94d9a7500533a/users/55ed193c93c446a8b373f770512e5160/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:00.279704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3112b4-1c98-47e0-9698-b80b9e0d34d4 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.289173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 794/1589] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.293760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacac71-321f-42d3-9a8f-7239b577a4d3 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.304682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacac71-321f-42d3-9a8f-7239b577a4d3 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.325085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:00.345281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3112b4-1c98-47e0-9698-b80b9e0d34d4 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.349994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ea25c-0c38-4257-bf97-97a838217bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.350663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ea25c-0c38-4257-bf97-97a838217bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.350950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ea25c-0c38-4257-bf97-97a838217bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.351226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ea25c-0c38-4257-bf97-97a838217bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.351955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ea25c-0c38-4257-bf97-97a838217bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.354190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ea25c-0c38-4257-bf97-97a838217bd2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.362773 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887a1b8-7935-4386-a401-6e559142716e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.363451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887a1b8-7935-4386-a401-6e559142716e admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.363757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887a1b8-7935-4386-a401-6e559142716e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.364033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887a1b8-7935-4386-a401-6e559142716e admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.367466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 796/1590] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.367655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887a1b8-7935-4386-a401-6e559142716e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.367747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ea25c-0c38-4257-bf97-97a838217bd2 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.367869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887a1b8-7935-4386-a401-6e559142716e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.375588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ea25c-0c38-4257-bf97-97a838217bd2 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.379662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 795/1591] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.379662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887a1b8-7935-4386-a401-6e559142716e admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.380744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e19e0-31f9-4644-a3f0-c1fce7e10510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.381430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e19e0-31f9-4644-a3f0-c1fce7e10510 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.386410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e19e0-31f9-4644-a3f0-c1fce7e10510 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.386410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e19e0-31f9-4644-a3f0-c1fce7e10510 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.386410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e19e0-31f9-4644-a3f0-c1fce7e10510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.386410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e19e0-31f9-4644-a3f0-c1fce7e10510 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.387263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887a1b8-7935-4386-a401-6e559142716e admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.391969 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af15919-5b76-49ee-8d04-896deab98b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.392528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af15919-5b76-49ee-8d04-896deab98b5e admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.392843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af15919-5b76-49ee-8d04-896deab98b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.393599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af15919-5b76-49ee-8d04-896deab98b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.393799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af15919-5b76-49ee-8d04-896deab98b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.395480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 797/1592] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.395709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af15919-5b76-49ee-8d04-896deab98b5e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.396516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e19e0-31f9-4644-a3f0-c1fce7e10510 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.407265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e19e0-31f9-4644-a3f0-c1fce7e10510 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.412309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a2025-39cd-405e-b542-85bb44b8a36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.415334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a2025-39cd-405e-b542-85bb44b8a36b admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.415334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a2025-39cd-405e-b542-85bb44b8a36b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.415334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a2025-39cd-405e-b542-85bb44b8a36b admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.415334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a2025-39cd-405e-b542-85bb44b8a36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.416428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a2025-39cd-405e-b542-85bb44b8a36b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.427646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 798/1593] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.428636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a2025-39cd-405e-b542-85bb44b8a36b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.438353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a2025-39cd-405e-b542-85bb44b8a36b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.442964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17513331-7f1c-46d8-a7bf-c8dca2f11911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.443636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17513331-7f1c-46d8-a7bf-c8dca2f11911 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.443925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17513331-7f1c-46d8-a7bf-c8dca2f11911 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.444202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17513331-7f1c-46d8-a7bf-c8dca2f11911 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.444953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17513331-7f1c-46d8-a7bf-c8dca2f11911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.446561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17513331-7f1c-46d8-a7bf-c8dca2f11911 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.456608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 799/1594] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.458013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6104c19-0623-47d8-8fb7-6e7f3e86b227 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.458306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6104c19-0623-47d8-8fb7-6e7f3e86b227 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.458585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6104c19-0623-47d8-8fb7-6e7f3e86b227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.476674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d6104c19-0623-47d8-8fb7-6e7f3e86b227 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:00.478512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d6104c19-0623-47d8-8fb7-6e7f3e86b227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:00.486139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 800/1595] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:00.486927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6104c19-0623-47d8-8fb7-6e7f3e86b227 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.491629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:00.494151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 796/1596] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:00 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:00.495774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af15919-5b76-49ee-8d04-896deab98b5e admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.497405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6104c19-0623-47d8-8fb7-6e7f3e86b227 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.555812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520089f-b6ea-4173-b293-6ed23a6d0d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.556618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520089f-b6ea-4173-b293-6ed23a6d0d9e admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.556930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520089f-b6ea-4173-b293-6ed23a6d0d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.557210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520089f-b6ea-4173-b293-6ed23a6d0d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.557945 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520089f-b6ea-4173-b293-6ed23a6d0d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.559612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520089f-b6ea-4173-b293-6ed23a6d0d9e admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.559871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af15919-5b76-49ee-8d04-896deab98b5e admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.564562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94aba61-c718-4de3-9953-a5e43c9f75fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.565266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94aba61-c718-4de3-9953-a5e43c9f75fa admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.565543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94aba61-c718-4de3-9953-a5e43c9f75fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.565812 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94aba61-c718-4de3-9953-a5e43c9f75fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.566533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94aba61-c718-4de3-9953-a5e43c9f75fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.568102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94aba61-c718-4de3-9953-a5e43c9f75fa admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.571957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 801/1597] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.573224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520089f-b6ea-4173-b293-6ed23a6d0d9e admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.583899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520089f-b6ea-4173-b293-6ed23a6d0d9e admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.584391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 797/1598] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.585258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94aba61-c718-4de3-9953-a5e43c9f75fa admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.593523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3215f89-c933-4286-8350-b6519426dd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.594424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3215f89-c933-4286-8350-b6519426dd8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.594711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3215f89-c933-4286-8350-b6519426dd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.594984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3215f89-c933-4286-8350-b6519426dd8b admin admin] PATH_INFO: `/v3/projects/60402d6f6eec43b1b215a506135c7496/users/ed7495b443ce48dd94b690b7c363e93c/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.598729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94aba61-c718-4de3-9953-a5e43c9f75fa admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.603393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5da9b-27dd-434c-9ed6-621ea47d2139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.604300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5da9b-27dd-434c-9ed6-621ea47d2139 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.604582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5da9b-27dd-434c-9ed6-621ea47d2139 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.604881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5da9b-27dd-434c-9ed6-621ea47d2139 admin admin] PATH_INFO: `/v3/projects/c1d92a82166d4e128dbef71394daba86/users/0602248c62664dd4a8809225d14ca5d7/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.611887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3215f89-c933-4286-8350-b6519426dd8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60402d6f6eec43b1b215a506135c7496, user_id=ed7495b443ce48dd94b690b7c363e93c, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.615158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3215f89-c933-4286-8350-b6519426dd8b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.619981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5da9b-27dd-434c-9ed6-621ea47d2139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d92a82166d4e128dbef71394daba86, user_id=0602248c62664dd4a8809225d14ca5d7, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.622636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5da9b-27dd-434c-9ed6-621ea47d2139 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.720284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 802/1599] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:00 2022] PUT /identity/v3/projects/60402d6f6eec43b1b215a506135c7496/users/ed7495b443ce48dd94b690b7c363e93c/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:00.721171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3215f89-c933-4286-8350-b6519426dd8b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.758791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 798/1600] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:00 2022] PUT /identity/v3/projects/c1d92a82166d4e128dbef71394daba86/users/0602248c62664dd4a8809225d14ca5d7/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:00.759726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5da9b-27dd-434c-9ed6-621ea47d2139 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.798554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3215f89-c933-4286-8350-b6519426dd8b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.831002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5da9b-27dd-434c-9ed6-621ea47d2139 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.835816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09037ce0-90ca-428d-9959-bd7467bc6f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.837111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09037ce0-90ca-428d-9959-bd7467bc6f2f admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.837399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09037ce0-90ca-428d-9959-bd7467bc6f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.837678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09037ce0-90ca-428d-9959-bd7467bc6f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.838407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09037ce0-90ca-428d-9959-bd7467bc6f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.840426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09037ce0-90ca-428d-9959-bd7467bc6f2f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.853749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 799/1601] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.853749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09037ce0-90ca-428d-9959-bd7467bc6f2f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.857860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdf465-e196-4345-a43f-009697d5ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.858545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf465-e196-4345-a43f-009697d5ed39 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.858835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf465-e196-4345-a43f-009697d5ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.859148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf465-e196-4345-a43f-009697d5ed39 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.859890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdf465-e196-4345-a43f-009697d5ed39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.860834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:00.861550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdf465-e196-4345-a43f-009697d5ed39 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.871744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 803/1602] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:00.874326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdf465-e196-4345-a43f-009697d5ed39 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.882993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdf465-e196-4345-a43f-009697d5ed39 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.888543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e26913-de6c-499e-9040-328368769578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.889246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e26913-de6c-499e-9040-328368769578 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.889534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e26913-de6c-499e-9040-328368769578 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.889812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e26913-de6c-499e-9040-328368769578 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.890503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e26913-de6c-499e-9040-328368769578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.892528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e26913-de6c-499e-9040-328368769578 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.909117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09037ce0-90ca-428d-9959-bd7467bc6f2f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.912856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efe920-5aeb-47dc-b5b8-acb9f5bca7ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.913508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efe920-5aeb-47dc-b5b8-acb9f5bca7ed service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.913799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efe920-5aeb-47dc-b5b8-acb9f5bca7ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.914080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efe920-5aeb-47dc-b5b8-acb9f5bca7ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.923486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efe920-5aeb-47dc-b5b8-acb9f5bca7ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.924952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efe920-5aeb-47dc-b5b8-acb9f5bca7ed service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.935017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 800/1603] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:00.935808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efe920-5aeb-47dc-b5b8-acb9f5bca7ed service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.944949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efe920-5aeb-47dc-b5b8-acb9f5bca7ed service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:00.950175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483a977-cfcf-44f2-a71e-5074fb5569ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:00.951077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483a977-cfcf-44f2-a71e-5074fb5569ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:00.951366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483a977-cfcf-44f2-a71e-5074fb5569ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:00.951649 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483a977-cfcf-44f2-a71e-5074fb5569ea admin admin] PATH_INFO: `/v3/projects/2e432ae974854f22b41d5c922fa4bd86/users/702d182b40104b329076ed297d91edb6/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:00.967248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483a977-cfcf-44f2-a71e-5074fb5569ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e432ae974854f22b41d5c922fa4bd86, user_id=702d182b40104b329076ed297d91edb6, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:00.969995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483a977-cfcf-44f2-a71e-5074fb5569ea admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:00.980874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:00.986271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 804/1604] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:00 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:00.987121 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e26913-de6c-499e-9040-328368769578 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:00.997492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e26913-de6c-499e-9040-328368769578 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:01.002083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddd2c4-c168-49e7-adf7-fbc9d1179c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:01.002990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddd2c4-c168-49e7-adf7-fbc9d1179c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.003230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddd2c4-c168-49e7-adf7-fbc9d1179c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.003507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddd2c4-c168-49e7-adf7-fbc9d1179c59 admin admin] PATH_INFO: `/v3/projects/e34a4f01af0a4015b20c52b6cfec5d40/users/447cd477e257464bbd138c87526e558b/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.018822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddd2c4-c168-49e7-adf7-fbc9d1179c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e34a4f01af0a4015b20c52b6cfec5d40, user_id=447cd477e257464bbd138c87526e558b, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:01.021489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddd2c4-c168-49e7-adf7-fbc9d1179c59 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:01.091847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 801/1605] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:00 2022] PUT /identity/v3/projects/2e432ae974854f22b41d5c922fa4bd86/users/702d182b40104b329076ed297d91edb6/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:01.093222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 805/1606] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:00 2022] PUT /identity/v3/projects/e34a4f01af0a4015b20c52b6cfec5d40/users/447cd477e257464bbd138c87526e558b/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:01.093412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e37aa-a96a-4a29-8527-59a3fe6d9b03 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.093798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e37aa-a96a-4a29-8527-59a3fe6d9b03 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.094107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e37aa-a96a-4a29-8527-59a3fe6d9b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.096126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb42e-2bb2-4aee-8b92-58d5b7bd84dc None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.096712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb42e-2bb2-4aee-8b92-58d5b7bd84dc None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.097171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb42e-2bb2-4aee-8b92-58d5b7bd84dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.116780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-9aedb42e-2bb2-4aee-8b92-58d5b7bd84dc None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:01.118949 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9aedb42e-2bb2-4aee-8b92-58d5b7bd84dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:01.129126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-622e37aa-a96a-4a29-8527-59a3fe6d9b03 None None] MFA Rules not processed for user `ed7495b443ce48dd94b690b7c363e93c`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:01.130358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-622e37aa-a96a-4a29-8527-59a3fe6d9b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:01.190874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 806/1607] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:01.191899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb42e-2bb2-4aee-8b92-58d5b7bd84dc None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:01.205224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb42e-2bb2-4aee-8b92-58d5b7bd84dc None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:01.210438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b0c2-5333-4d53-9730-2dfc31187980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:01.211381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b0c2-5333-4d53-9730-2dfc31187980 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.211523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 802/1608] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:01.211675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b0c2-5333-4d53-9730-2dfc31187980 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.211958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b0c2-5333-4d53-9730-2dfc31187980 admin admin] PATH_INFO: `/v3/projects/15c454baefdc474682f88c603f6108a8/users/5221431e0552435d9ff82f8ce75b624e/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.212210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e37aa-a96a-4a29-8527-59a3fe6d9b03 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:01.225024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e37aa-a96a-4a29-8527-59a3fe6d9b03 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:01.229932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf7198-de28-4617-a397-f5ccb5da8d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:01.230832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf7198-de28-4617-a397-f5ccb5da8d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.231175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf7198-de28-4617-a397-f5ccb5da8d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.231392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf7198-de28-4617-a397-f5ccb5da8d8a admin admin] PATH_INFO: `/v3/projects/b6b7c24ef86e420fb3e454ef5a185551/users/249bb2916e1a4257b8c2aeb8eee99191/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.232333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8b0c2-5333-4d53-9730-2dfc31187980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c454baefdc474682f88c603f6108a8, user_id=5221431e0552435d9ff82f8ce75b624e, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:01.235883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8b0c2-5333-4d53-9730-2dfc31187980 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:01.249011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf7198-de28-4617-a397-f5ccb5da8d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b7c24ef86e420fb3e454ef5a185551, user_id=249bb2916e1a4257b8c2aeb8eee99191, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:01.251725 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf7198-de28-4617-a397-f5ccb5da8d8a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:01.315336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 807/1609] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:01 2022] PUT /identity/v3/projects/15c454baefdc474682f88c603f6108a8/users/5221431e0552435d9ff82f8ce75b624e/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:01.316964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b0c2-5333-4d53-9730-2dfc31187980 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:01.377556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 803/1610] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:01 2022] PUT /identity/v3/projects/b6b7c24ef86e420fb3e454ef5a185551/users/249bb2916e1a4257b8c2aeb8eee99191/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:01.379561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a73ee-80b3-4426-b3ca-56b335c23fc3 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.379855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a73ee-80b3-4426-b3ca-56b335c23fc3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.380141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a73ee-80b3-4426-b3ca-56b335c23fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.396606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b0c2-5333-4d53-9730-2dfc31187980 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:01.420215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e21a73ee-80b3-4426-b3ca-56b335c23fc3 None None] MFA Rules not processed for user `702d182b40104b329076ed297d91edb6`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:01.421783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e21a73ee-80b3-4426-b3ca-56b335c23fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:01.470167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a55451-6ab8-4b13-8124-3d1472ec540a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:01.470816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a55451-6ab8-4b13-8124-3d1472ec540a admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.471282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a55451-6ab8-4b13-8124-3d1472ec540a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.471559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a55451-6ab8-4b13-8124-3d1472ec540a admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.472459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a55451-6ab8-4b13-8124-3d1472ec540a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:01.473813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:01.474940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a55451-6ab8-4b13-8124-3d1472ec540a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:01.476601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:01.488904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 808/1611] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:01.490854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5585-3e52-4740-9614-2e14094836f8 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.491295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5585-3e52-4740-9614-2e14094836f8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.491586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5585-3e52-4740-9614-2e14094836f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.503704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 804/1612] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:01.505573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a73ee-80b3-4426-b3ca-56b335c23fc3 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:01.525287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-964e5585-3e52-4740-9614-2e14094836f8 None None] MFA Rules not processed for user `447cd477e257464bbd138c87526e558b`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:01.526707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-964e5585-3e52-4740-9614-2e14094836f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:01.576979 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a73ee-80b3-4426-b3ca-56b335c23fc3 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:01.580715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018edf6c-1a97-48d4-a283-9288b7898468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:01.581370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018edf6c-1a97-48d4-a283-9288b7898468 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.581800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018edf6c-1a97-48d4-a283-9288b7898468 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.582082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018edf6c-1a97-48d4-a283-9288b7898468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.592668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018edf6c-1a97-48d4-a283-9288b7898468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:01.594175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018edf6c-1a97-48d4-a283-9288b7898468 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:01.602918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 809/1613] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:01.603822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e5585-3e52-4740-9614-2e14094836f8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:01.604869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 805/1614] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:01.606037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a93dcd-a794-48ad-9060-a52330234b6f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.606518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a93dcd-a794-48ad-9060-a52330234b6f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.606810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a93dcd-a794-48ad-9060-a52330234b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.615195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e5585-3e52-4740-9614-2e14094836f8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:01.620706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72131545-4cc7-42cd-8edb-3a3d046638f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:01.621498 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72131545-4cc7-42cd-8edb-3a3d046638f7 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.621791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72131545-4cc7-42cd-8edb-3a3d046638f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.622077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72131545-4cc7-42cd-8edb-3a3d046638f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.622785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72131545-4cc7-42cd-8edb-3a3d046638f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:01.624941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72131545-4cc7-42cd-8edb-3a3d046638f7 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:01.628873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-07a93dcd-a794-48ad-9060-a52330234b6f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:01.629854 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-07a93dcd-a794-48ad-9060-a52330234b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:01.638093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 806/1615] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:01.640080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aaffcb-7ff7-4f13-bb92-818c4e2b9fe4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.641285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aaffcb-7ff7-4f13-bb92-818c4e2b9fe4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.641285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aaffcb-7ff7-4f13-bb92-818c4e2b9fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.667424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f9aaffcb-7ff7-4f13-bb92-818c4e2b9fe4 None None] MFA Rules not processed for user `249bb2916e1a4257b8c2aeb8eee99191`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:01.669960 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f9aaffcb-7ff7-4f13-bb92-818c4e2b9fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:01.748205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 807/1616] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:01.749099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aaffcb-7ff7-4f13-bb92-818c4e2b9fe4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:01.762329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aaffcb-7ff7-4f13-bb92-818c4e2b9fe4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:01.764479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 810/1617] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:01 2022] POST /identity/v3/projects => generated 369 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:01.765521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72131545-4cc7-42cd-8edb-3a3d046638f7 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:01.832456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967a110-1610-402a-b6d8-b80868c1d536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:01.833411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967a110-1610-402a-b6d8-b80868c1d536 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.833720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967a110-1610-402a-b6d8-b80868c1d536 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.834367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967a110-1610-402a-b6d8-b80868c1d536 admin admin] PATH_INFO: `/v3/projects/d35c4a2b642949658935d4da44b298e4/users/526ce1eb4acd4fcca84e59abb0ab0bdd/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.841989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72131545-4cc7-42cd-8edb-3a3d046638f7 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:01.847342 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:01.848281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450e9fd-d404-470b-98e4-608453ba91a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:01.849007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e9fd-d404-470b-98e4-608453ba91a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:01.849300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e9fd-d404-470b-98e4-608453ba91a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:01.849591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e9fd-d404-470b-98e4-608453ba91a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:01.850306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450e9fd-d404-470b-98e4-608453ba91a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:01.852719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967a110-1610-402a-b6d8-b80868c1d536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d35c4a2b642949658935d4da44b298e4, user_id=526ce1eb4acd4fcca84e59abb0ab0bdd, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:01.852984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450e9fd-d404-470b-98e4-608453ba91a7 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:01.855604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967a110-1610-402a-b6d8-b80868c1d536 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:01.959669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 808/1618] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:01 2022] PUT /identity/v3/projects/d35c4a2b642949658935d4da44b298e4/users/526ce1eb4acd4fcca84e59abb0ab0bdd/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:01.964491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967a110-1610-402a-b6d8-b80868c1d536 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:01.964611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 811/1619] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:01 2022] POST /identity/v3/projects => generated 357 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:01.964611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450e9fd-d404-470b-98e4-608453ba91a7 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.030352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:02.034216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967a110-1610-402a-b6d8-b80868c1d536 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.035108 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450e9fd-d404-470b-98e4-608453ba91a7 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.038841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f6dc7-a424-4319-805b-faedb0b695a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.040109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f6dc7-a424-4319-805b-faedb0b695a8 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.040519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f6dc7-a424-4319-805b-faedb0b695a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.040866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f6dc7-a424-4319-805b-faedb0b695a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.114396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f6dc7-a424-4319-805b-faedb0b695a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.115867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f6dc7-a424-4319-805b-faedb0b695a8 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.124867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806f1ec-80ec-4580-8012-6a4aec757bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.125946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806f1ec-80ec-4580-8012-6a4aec757bac admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.126437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806f1ec-80ec-4580-8012-6a4aec757bac admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.126910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 812/1620] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:02.127814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806f1ec-80ec-4580-8012-6a4aec757bac admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.127952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f6dc7-a424-4319-805b-faedb0b695a8 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.128034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806f1ec-80ec-4580-8012-6a4aec757bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.130993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806f1ec-80ec-4580-8012-6a4aec757bac admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.191373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f6dc7-a424-4319-805b-faedb0b695a8 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.196861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94707f1-8d83-4a77-99b1-0064b570f1cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.197583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94707f1-8d83-4a77-99b1-0064b570f1cf service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.197878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94707f1-8d83-4a77-99b1-0064b570f1cf service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.198183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94707f1-8d83-4a77-99b1-0064b570f1cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.216870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 809/1621] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:01 2022] POST /identity/v3/projects => generated 405 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:02.216870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806f1ec-80ec-4580-8012-6a4aec757bac admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.267554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94707f1-8d83-4a77-99b1-0064b570f1cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.269073 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94707f1-8d83-4a77-99b1-0064b570f1cf service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.278933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 813/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:02.279549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94707f1-8d83-4a77-99b1-0064b570f1cf service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.280901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806f1ec-80ec-4580-8012-6a4aec757bac admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.285982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c519cb3-86ae-416e-8547-5efc4b8e7577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.286667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c519cb3-86ae-416e-8547-5efc4b8e7577 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.287132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c519cb3-86ae-416e-8547-5efc4b8e7577 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.287336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c519cb3-86ae-416e-8547-5efc4b8e7577 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.288051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c519cb3-86ae-416e-8547-5efc4b8e7577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.298625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c519cb3-86ae-416e-8547-5efc4b8e7577 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.298896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94707f1-8d83-4a77-99b1-0064b570f1cf service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.307233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7288b-482c-4adc-8ebe-51493c8fc003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.308016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7288b-482c-4adc-8ebe-51493c8fc003 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.308931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7288b-482c-4adc-8ebe-51493c8fc003 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.309204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7288b-482c-4adc-8ebe-51493c8fc003 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.310014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7288b-482c-4adc-8ebe-51493c8fc003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.312042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7288b-482c-4adc-8ebe-51493c8fc003 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.328404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 814/1623] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:02.328404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7288b-482c-4adc-8ebe-51493c8fc003 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.344502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7288b-482c-4adc-8ebe-51493c8fc003 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.349877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82631c25-77af-4919-807f-a50791da611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.350704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82631c25-77af-4919-807f-a50791da611a admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.351085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82631c25-77af-4919-807f-a50791da611a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.351447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82631c25-77af-4919-807f-a50791da611a admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.352526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82631c25-77af-4919-807f-a50791da611a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.354506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82631c25-77af-4919-807f-a50791da611a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.461021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 810/1624] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:02 2022] POST /identity/v3/users => generated 291 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:02.462563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c519cb3-86ae-416e-8547-5efc4b8e7577 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.478940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c519cb3-86ae-416e-8547-5efc4b8e7577 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.487211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 815/1625] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:02 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:02.488280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82631c25-77af-4919-807f-a50791da611a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.491097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8e007-5d18-479a-a244-8280622bfbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.491841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8e007-5d18-479a-a244-8280622bfbcd admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.492129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8e007-5d18-479a-a244-8280622bfbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.492411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8e007-5d18-479a-a244-8280622bfbcd admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.493669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8e007-5d18-479a-a244-8280622bfbcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.496011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8e007-5d18-479a-a244-8280622bfbcd admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.502153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82631c25-77af-4919-807f-a50791da611a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.508192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56d151-e3d6-44ce-acf3-dd165d2cff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.512068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56d151-e3d6-44ce-acf3-dd165d2cff06 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.512068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56d151-e3d6-44ce-acf3-dd165d2cff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.512068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56d151-e3d6-44ce-acf3-dd165d2cff06 admin admin] PATH_INFO: `/v3/projects/d35c4a2b642949658935d4da44b298e4/users/526ce1eb4acd4fcca84e59abb0ab0bdd/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.514111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56d151-e3d6-44ce-acf3-dd165d2cff06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d35c4a2b642949658935d4da44b298e4, user_id=526ce1eb4acd4fcca84e59abb0ab0bdd, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.514623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:02.518841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56d151-e3d6-44ce-acf3-dd165d2cff06 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.543739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 816/1626] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:02 2022] PUT /identity/v3/projects/d35c4a2b642949658935d4da44b298e4/users/526ce1eb4acd4fcca84e59abb0ab0bdd/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:02.543739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56d151-e3d6-44ce-acf3-dd165d2cff06 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.565650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:02.589465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 811/1627] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:02 2022] POST /identity/v3/users => generated 309 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:02.590568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8e007-5d18-479a-a244-8280622bfbcd admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.612346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8e007-5d18-479a-a244-8280622bfbcd admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.657325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56d151-e3d6-44ce-acf3-dd165d2cff06 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.667567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09f8be-5366-4ab2-a74e-2ff2208dac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.669649 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09f8be-5366-4ab2-a74e-2ff2208dac37 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.670423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09f8be-5366-4ab2-a74e-2ff2208dac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.670423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09f8be-5366-4ab2-a74e-2ff2208dac37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.673180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09f8be-5366-4ab2-a74e-2ff2208dac37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.675371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09f8be-5366-4ab2-a74e-2ff2208dac37 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.700846 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 817/1628] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:02.704984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99391fa7-c88a-4151-b498-f255199a6b0e None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.706134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99391fa7-c88a-4151-b498-f255199a6b0e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.706685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99391fa7-c88a-4151-b498-f255199a6b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.745645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ba734-cffb-4164-84fe-f0fe2428a330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.745645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ba734-cffb-4164-84fe-f0fe2428a330 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.745645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ba734-cffb-4164-84fe-f0fe2428a330 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.745645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ba734-cffb-4164-84fe-f0fe2428a330 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.746194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ba734-cffb-4164-84fe-f0fe2428a330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.746194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ba734-cffb-4164-84fe-f0fe2428a330 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.770894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 812/1629] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:02.773354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ba734-cffb-4164-84fe-f0fe2428a330 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.783030 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-99391fa7-c88a-4151-b498-f255199a6b0e None None] MFA Rules not processed for user `526ce1eb4acd4fcca84e59abb0ab0bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:02.784515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-99391fa7-c88a-4151-b498-f255199a6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:02.785723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ba734-cffb-4164-84fe-f0fe2428a330 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.793454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9f200-eb8e-4899-ba50-61500f82e76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.793717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9f200-eb8e-4899-ba50-61500f82e76f admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.795035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9f200-eb8e-4899-ba50-61500f82e76f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.795035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9f200-eb8e-4899-ba50-61500f82e76f admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.796106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9f200-eb8e-4899-ba50-61500f82e76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.798650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9f200-eb8e-4899-ba50-61500f82e76f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.816315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 813/1630] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:02.817421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9f200-eb8e-4899-ba50-61500f82e76f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.836820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9f200-eb8e-4899-ba50-61500f82e76f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.851628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668cd41-2ff5-435b-ae24-a61e73c0c95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.853413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668cd41-2ff5-435b-ae24-a61e73c0c95f admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.853836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668cd41-2ff5-435b-ae24-a61e73c0c95f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.854214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668cd41-2ff5-435b-ae24-a61e73c0c95f admin admin] PATH_INFO: `/v3/projects/bd9a62a6eaaa48a288c6d76bc368f0ba/users/0b8877be001d4db784d9ae8c1d4fed0f/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:02.885301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668cd41-2ff5-435b-ae24-a61e73c0c95f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9a62a6eaaa48a288c6d76bc368f0ba, user_id=0b8877be001d4db784d9ae8c1d4fed0f, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:02.893181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668cd41-2ff5-435b-ae24-a61e73c0c95f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:02.921190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 818/1631] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:02.922572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99391fa7-c88a-4151-b498-f255199a6b0e None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:02.944016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99391fa7-c88a-4151-b498-f255199a6b0e None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:02.966563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e421a7-6d33-42f0-9718-16f645013d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:02.971592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e421a7-6d33-42f0-9718-16f645013d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:02.971592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e421a7-6d33-42f0-9718-16f645013d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:02.971592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e421a7-6d33-42f0-9718-16f645013d5d admin admin] PATH_INFO: `/v3/projects/99606e335add4c5bb3573651418a298a/users/b4d7c8ea41384cb5809e15faa3edfb3e/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.017222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e421a7-6d33-42f0-9718-16f645013d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99606e335add4c5bb3573651418a298a, user_id=b4d7c8ea41384cb5809e15faa3edfb3e, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:03.020681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e421a7-6d33-42f0-9718-16f645013d5d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:03.034690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 814/1632] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:02 2022] PUT /identity/v3/projects/bd9a62a6eaaa48a288c6d76bc368f0ba/users/0b8877be001d4db784d9ae8c1d4fed0f/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:03.036446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668cd41-2ff5-435b-ae24-a61e73c0c95f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:03.090103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 819/1633] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:02 2022] PUT /identity/v3/projects/99606e335add4c5bb3573651418a298a/users/b4d7c8ea41384cb5809e15faa3edfb3e/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:03.093511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e421a7-6d33-42f0-9718-16f645013d5d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:03.165788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668cd41-2ff5-435b-ae24-a61e73c0c95f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:03.197209 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e421a7-6d33-42f0-9718-16f645013d5d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:03.202789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf984fc-06b1-4e2a-8a68-c2d01e9272c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:03.204444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf984fc-06b1-4e2a-8a68-c2d01e9272c3 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:03.205009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf984fc-06b1-4e2a-8a68-c2d01e9272c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:03.205477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf984fc-06b1-4e2a-8a68-c2d01e9272c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.240936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:03.286270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a02c6-8dd2-4194-8fa1-9a943e9f60cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:03.287467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a02c6-8dd2-4194-8fa1-9a943e9f60cf admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:03.287467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a02c6-8dd2-4194-8fa1-9a943e9f60cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:03.290509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a02c6-8dd2-4194-8fa1-9a943e9f60cf admin admin] PATH_INFO: `/v3/projects/988327793be147e68671b64f413d0122/users/6074b20a159b446dab1e7db8a206139f/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.294906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf984fc-06b1-4e2a-8a68-c2d01e9272c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:03.296949 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf984fc-06b1-4e2a-8a68-c2d01e9272c3 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:03.297733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:03.315484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a02c6-8dd2-4194-8fa1-9a943e9f60cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=988327793be147e68671b64f413d0122, user_id=6074b20a159b446dab1e7db8a206139f, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:03.318410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a02c6-8dd2-4194-8fa1-9a943e9f60cf admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:03.324835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 820/1634] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:03.326115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf984fc-06b1-4e2a-8a68-c2d01e9272c3 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:03.339241 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf984fc-06b1-4e2a-8a68-c2d01e9272c3 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:03.345243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8fbf1-c613-4015-bdb3-a52ba56ffc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:03.346353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8fbf1-c613-4015-bdb3-a52ba56ffc79 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:03.346802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8fbf1-c613-4015-bdb3-a52ba56ffc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:03.347331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8fbf1-c613-4015-bdb3-a52ba56ffc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.348484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8fbf1-c613-4015-bdb3-a52ba56ffc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:03.350599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8fbf1-c613-4015-bdb3-a52ba56ffc79 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:03.366842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 821/1635] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:03.366842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8fbf1-c613-4015-bdb3-a52ba56ffc79 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:03.379546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8fbf1-c613-4015-bdb3-a52ba56ffc79 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:03.384542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf85f1-d27f-42cf-ba03-b32a7b738235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:03.385453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf85f1-d27f-42cf-ba03-b32a7b738235 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:03.391404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf85f1-d27f-42cf-ba03-b32a7b738235 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:03.391404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf85f1-d27f-42cf-ba03-b32a7b738235 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.391404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf85f1-d27f-42cf-ba03-b32a7b738235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:03.391404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf85f1-d27f-42cf-ba03-b32a7b738235 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:03.404194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 822/1636] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:03.405300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf85f1-d27f-42cf-ba03-b32a7b738235 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:03.431958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 815/1637] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:03 2022] PUT /identity/v3/projects/988327793be147e68671b64f413d0122/users/6074b20a159b446dab1e7db8a206139f/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:03.434264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a02c6-8dd2-4194-8fa1-9a943e9f60cf admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:03.438159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf85f1-d27f-42cf-ba03-b32a7b738235 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:03.555275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd1647-97da-411c-9489-605a6964de34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:03.556296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd1647-97da-411c-9489-605a6964de34 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:03.556676 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd1647-97da-411c-9489-605a6964de34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:03.557062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd1647-97da-411c-9489-605a6964de34 admin admin] PATH_INFO: `/v3/projects/bd9a62a6eaaa48a288c6d76bc368f0ba/users/0b8877be001d4db784d9ae8c1d4fed0f/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.559461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd1647-97da-411c-9489-605a6964de34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9a62a6eaaa48a288c6d76bc368f0ba, user_id=0b8877be001d4db784d9ae8c1d4fed0f, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:03.561187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a02c6-8dd2-4194-8fa1-9a943e9f60cf admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:03.562659 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd1647-97da-411c-9489-605a6964de34 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:03.578465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058676f5-d000-4ff9-ac8f-8ba6a813ff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:03.579311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058676f5-d000-4ff9-ac8f-8ba6a813ff2e admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:03.579597 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058676f5-d000-4ff9-ac8f-8ba6a813ff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:03.580048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058676f5-d000-4ff9-ac8f-8ba6a813ff2e admin admin] PATH_INFO: `/v3/projects/99606e335add4c5bb3573651418a298a/users/b4d7c8ea41384cb5809e15faa3edfb3e/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.582149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058676f5-d000-4ff9-ac8f-8ba6a813ff2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99606e335add4c5bb3573651418a298a, user_id=b4d7c8ea41384cb5809e15faa3edfb3e, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:03.583970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 823/1638] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:03 2022] PUT /identity/v3/projects/bd9a62a6eaaa48a288c6d76bc368f0ba/users/0b8877be001d4db784d9ae8c1d4fed0f/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:03.585430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd1647-97da-411c-9489-605a6964de34 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:03.585917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058676f5-d000-4ff9-ac8f-8ba6a813ff2e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:03.615455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 816/1639] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:03 2022] PUT /identity/v3/projects/99606e335add4c5bb3573651418a298a/users/b4d7c8ea41384cb5809e15faa3edfb3e/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:03.615455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d11d90-1f37-43e5-9e25-3b8081871190 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:03.615455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d11d90-1f37-43e5-9e25-3b8081871190 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:03.615455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d11d90-1f37-43e5-9e25-3b8081871190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.687984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d9d11d90-1f37-43e5-9e25-3b8081871190 None None] MFA Rules not processed for user `0b8877be001d4db784d9ae8c1d4fed0f`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:03.692541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d9d11d90-1f37-43e5-9e25-3b8081871190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:03.713686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd1647-97da-411c-9489-605a6964de34 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:03.814708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 817/1640] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:03.816464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39371928-884a-4b1f-9d86-f993ef3d15b9 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:03.817176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39371928-884a-4b1f-9d86-f993ef3d15b9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:03.817472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39371928-884a-4b1f-9d86-f993ef3d15b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.888030 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64d93e-abd4-432d-9f9b-03598a5f0bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:03.892174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64d93e-abd4-432d-9f9b-03598a5f0bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:03.893901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64d93e-abd4-432d-9f9b-03598a5f0bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:03.894888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64d93e-abd4-432d-9f9b-03598a5f0bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:03.897658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64d93e-abd4-432d-9f9b-03598a5f0bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:03.903100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64d93e-abd4-432d-9f9b-03598a5f0bb8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:03.911562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-39371928-884a-4b1f-9d86-f993ef3d15b9 None None] MFA Rules not processed for user `b4d7c8ea41384cb5809e15faa3edfb3e`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:03.916246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-39371928-884a-4b1f-9d86-f993ef3d15b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:03.924371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:03.937140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 824/1641] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:03.937140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64d93e-abd4-432d-9f9b-03598a5f0bb8 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:04.086524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:04.102935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64d93e-abd4-432d-9f9b-03598a5f0bb8 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:04.108813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc6c5b-aca2-4f37-9b98-be23ae91f531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:04.110538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc6c5b-aca2-4f37-9b98-be23ae91f531 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:04.111495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc6c5b-aca2-4f37-9b98-be23ae91f531 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:04.111495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc6c5b-aca2-4f37-9b98-be23ae91f531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:04.134437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 818/1642] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:03 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:04.134437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39371928-884a-4b1f-9d86-f993ef3d15b9 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:04.156739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39371928-884a-4b1f-9d86-f993ef3d15b9 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:04.170503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc6c5b-aca2-4f37-9b98-be23ae91f531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:04.172308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc6c5b-aca2-4f37-9b98-be23ae91f531 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:04.186347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40df62-34b2-4792-b9a3-eda3b10fb7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:04.190919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40df62-34b2-4792-b9a3-eda3b10fb7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:04.190919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40df62-34b2-4792-b9a3-eda3b10fb7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:04.190919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40df62-34b2-4792-b9a3-eda3b10fb7bc admin admin] PATH_INFO: `/v3/projects/988327793be147e68671b64f413d0122/users/6074b20a159b446dab1e7db8a206139f/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:04.200602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40df62-34b2-4792-b9a3-eda3b10fb7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=988327793be147e68671b64f413d0122, user_id=6074b20a159b446dab1e7db8a206139f, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:04.206620 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40df62-34b2-4792-b9a3-eda3b10fb7bc admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:04.221967 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 825/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:03 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:04.489472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 819/1644] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:04 2022] PUT /identity/v3/projects/988327793be147e68671b64f413d0122/users/6074b20a159b446dab1e7db8a206139f/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:04.731501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5d15-fd10-4fac-a272-9c600269f57d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:04.731501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5d15-fd10-4fac-a272-9c600269f57d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:04.731501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5d15-fd10-4fac-a272-9c600269f57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:04.731501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-922e5d15-fd10-4fac-a272-9c600269f57d None None] MFA Rules not processed for user `6074b20a159b446dab1e7db8a206139f`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:04.731501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-922e5d15-fd10-4fac-a272-9c600269f57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:04.737147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40df62-34b2-4792-b9a3-eda3b10fb7bc admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:04.737947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 826/1645] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:04 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:04.738558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40df62-34b2-4792-b9a3-eda3b10fb7bc admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:04.738558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75e3b-87c0-4dbe-90d8-ac02d5517fcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:04.738558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75e3b-87c0-4dbe-90d8-ac02d5517fcd service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:04.738558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75e3b-87c0-4dbe-90d8-ac02d5517fcd service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:04.739540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75e3b-87c0-4dbe-90d8-ac02d5517fcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:04.739540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a75e3b-87c0-4dbe-90d8-ac02d5517fcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:04.739540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a75e3b-87c0-4dbe-90d8-ac02d5517fcd service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:04.746324 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 820/1646] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:04 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:04.919476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e5d15-fd10-4fac-a272-9c600269f57d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:04.929814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e5d15-fd10-4fac-a272-9c600269f57d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:04.942182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d18ba-fbef-4166-a33c-fad140032ba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:04.942917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d18ba-fbef-4166-a33c-fad140032ba1 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:04.944497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d18ba-fbef-4166-a33c-fad140032ba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:04.944497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d18ba-fbef-4166-a33c-fad140032ba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:05.007709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75e3b-87c0-4dbe-90d8-ac02d5517fcd service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:05.020094 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75e3b-87c0-4dbe-90d8-ac02d5517fcd service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:05.031281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765cc4e-1648-4f86-b4e4-2e025b80ce2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:05.032119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765cc4e-1648-4f86-b4e4-2e025b80ce2c service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:05.032411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765cc4e-1648-4f86-b4e4-2e025b80ce2c service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:05.041648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765cc4e-1648-4f86-b4e4-2e025b80ce2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:05.071656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765cc4e-1648-4f86-b4e4-2e025b80ce2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:05.073424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765cc4e-1648-4f86-b4e4-2e025b80ce2c service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:05.107250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d18ba-fbef-4166-a33c-fad140032ba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:05.108516 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 821/1647] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:05 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:01:05.110438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d18ba-fbef-4166-a33c-fad140032ba1 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:05.166386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 827/1648] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:05.695980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765cc4e-1648-4f86-b4e4-2e025b80ce2c service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:05.706311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765cc4e-1648-4f86-b4e4-2e025b80ce2c service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:05.717219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce38da4-ccc6-4fe0-90e5-96900a6a54a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:05.717902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38da4-ccc6-4fe0-90e5-96900a6a54a3 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:05.718197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38da4-ccc6-4fe0-90e5-96900a6a54a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:05.719315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38da4-ccc6-4fe0-90e5-96900a6a54a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:05.867812 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce38da4-ccc6-4fe0-90e5-96900a6a54a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:05.870253 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce38da4-ccc6-4fe0-90e5-96900a6a54a3 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:05.884248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 822/1649] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:05 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:05.941482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae911ef6-6bf2-4549-9bcf-391d3958c35d None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:05.941995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae911ef6-6bf2-4549-9bcf-391d3958c35d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:05.942517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae911ef6-6bf2-4549-9bcf-391d3958c35d None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:05.947806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 828/1650] 10.222.0.17 () {54 vars in 962 bytes} [Sun Jul 24 20:01:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:05.992236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f32435-306c-4bd4-87ce-0f8d904fa50e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:05.993057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f32435-306c-4bd4-87ce-0f8d904fa50e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:05.995171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f32435-306c-4bd4-87ce-0f8d904fa50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:06.078848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:06.126226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-00f32435-306c-4bd4-87ce-0f8d904fa50e None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:06.129428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-00f32435-306c-4bd4-87ce-0f8d904fa50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:06.238476 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 823/1651] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Jul 24 20:01:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:06.247251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a19d-5c84-447a-a229-a37e63bc2854 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:06.248789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a19d-5c84-447a-a229-a37e63bc2854 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:06.249225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a19d-5c84-447a-a229-a37e63bc2854 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:06.252314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 829/1652] 10.222.0.17 () {54 vars in 912 bytes} [Sun Jul 24 20:01:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:06.263050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04e33a-fcb4-447a-ad44-ef2b46ae8988 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:06.263591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04e33a-fcb4-447a-ad44-ef2b46ae8988 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:06.264056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04e33a-fcb4-447a-ad44-ef2b46ae8988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:06.290201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6a04e33a-fcb4-447a-ad44-ef2b46ae8988 None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:06.292261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6a04e33a-fcb4-447a-ad44-ef2b46ae8988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:06.315410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 824/1653] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:01:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:06.434173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a19d-5c84-447a-a229-a37e63bc2854 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:06.442130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:06.450888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a19d-5c84-447a-a229-a37e63bc2854 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:06.464601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bfc11-68d1-49dd-833a-2008dc4662ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:06.465306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bfc11-68d1-49dd-833a-2008dc4662ce service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:06.465598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bfc11-68d1-49dd-833a-2008dc4662ce service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:06.465885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bfc11-68d1-49dd-833a-2008dc4662ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:06.495575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bfc11-68d1-49dd-833a-2008dc4662ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:06.496460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bfc11-68d1-49dd-833a-2008dc4662ce service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:06.517695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 830/1654] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:06.751318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04e33a-fcb4-447a-ad44-ef2b46ae8988 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:06.970873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04e33a-fcb4-447a-ad44-ef2b46ae8988 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:06.987075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:06.987996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:06.989044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:06.989333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] PATH_INFO: `/v3/users/447cd477e257464bbd138c87526e558b` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:06.990771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] RBAC: Authorizing `identity:delete_user(user_id=447cd477e257464bbd138c87526e558b)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:06.992797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:07.349209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '447cd477e257464bbd138c87526e558b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:07.377482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '447cd477e257464bbd138c87526e558b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:07.378435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '447cd477e257464bbd138c87526e558b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:07.471698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '447cd477e257464bbd138c87526e558b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:07.502260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '447cd477e257464bbd138c87526e558b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:07.514064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 825/1655] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:06 2022] DELETE /identity/v3/users/447cd477e257464bbd138c87526e558b => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:07.518980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b41c36-5dfe-4f21-ba2e-0391e1986a50 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:07.519516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b41c36-5dfe-4f21-ba2e-0391e1986a50 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:07.519749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b41c36-5dfe-4f21-ba2e-0391e1986a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:07.574754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-00b41c36-5dfe-4f21-ba2e-0391e1986a50 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:07.576453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-00b41c36-5dfe-4f21-ba2e-0391e1986a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:07.797950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 831/1656] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:07.857031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:07.859169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:08.015184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c44600-2be4-44d2-97d9-74fc1f23605d admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:08.032740 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b586ce9-a592-4ac1-b0e0-be538d0dd825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:08.038993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b586ce9-a592-4ac1-b0e0-be538d0dd825 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:08.043818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b586ce9-a592-4ac1-b0e0-be538d0dd825 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:08.044310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b586ce9-a592-4ac1-b0e0-be538d0dd825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:08.098986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b586ce9-a592-4ac1-b0e0-be538d0dd825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:08.103438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b586ce9-a592-4ac1-b0e0-be538d0dd825 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:08.159976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 826/1657] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:08.245336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:09.233338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b41c36-5dfe-4f21-ba2e-0391e1986a50 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:09.235366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-00b41c36-5dfe-4f21-ba2e-0391e1986a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:09.276842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b41c36-5dfe-4f21-ba2e-0391e1986a50 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:09.293344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:09.294236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:09.294871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:09.295304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] PATH_INFO: `/v3/projects/e34a4f01af0a4015b20c52b6cfec5d40` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:09.300565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e34a4f01af0a4015b20c52b6cfec5d40)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:09.302892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:09.552057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e34a4f01af0a4015b20c52b6cfec5d40', 'request_id': 'req-d7055bbc-a07b-488c-8d6c-ec0155eff235'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:09.559535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e34a4f01af0a4015b20c52b6cfec5d40', 'request_id': 'req-d7055bbc-a07b-488c-8d6c-ec0155eff235'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:09.569401 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e34a4f01af0a4015b20c52b6cfec5d40', 'request_id': 'req-d7055bbc-a07b-488c-8d6c-ec0155eff235'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:09.634355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] The token cache is being invalidate because project e34a4f01af0a4015b20c52b6cfec5d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:09.634658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:09.636453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:09.637967 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 832/1658] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:09 2022] DELETE /identity/v3/projects/e34a4f01af0a4015b20c52b6cfec5d40 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:09.641849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b586ce9-a592-4ac1-b0e0-be538d0dd825 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:09.652176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b586ce9-a592-4ac1-b0e0-be538d0dd825 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:09.849611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:09.851010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:09.851494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:09.851993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] PATH_INFO: `/v3/users/6074b20a159b446dab1e7db8a206139f` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:09.855422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6074b20a159b446dab1e7db8a206139f)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:09.858098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:10.169650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6074b20a159b446dab1e7db8a206139f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:10.190774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6074b20a159b446dab1e7db8a206139f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:10.191834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6074b20a159b446dab1e7db8a206139f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:10.346616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6074b20a159b446dab1e7db8a206139f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:10.485670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b7269d97-f9d1-4222-a768-7476b523bba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6074b20a159b446dab1e7db8a206139f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:10.485670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 827/1659] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:09 2022] DELETE /identity/v3/users/6074b20a159b446dab1e7db8a206139f => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:11.473505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:11.475690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:11.504989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7055bbc-a07b-488c-8d6c-ec0155eff235 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:11.709037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:11.710105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:11.710399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:11.710876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] PATH_INFO: `/v3/projects/988327793be147e68671b64f413d0122` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:11.712909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=988327793be147e68671b64f413d0122)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:11.719482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:11.799360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:11.925306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988327793be147e68671b64f413d0122', 'request_id': 'req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:11.950216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988327793be147e68671b64f413d0122', 'request_id': 'req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:11.971852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988327793be147e68671b64f413d0122', 'request_id': 'req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:12.084311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] The token cache is being invalidate because project 988327793be147e68671b64f413d0122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:12.084887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:12.085813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a60aac6c-3bc2-47c3-a6e0-6626c0c1bcb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:12.088720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 833/1660] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:11 2022] DELETE /identity/v3/projects/988327793be147e68671b64f413d0122 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:12.093698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e080c8-784b-4b8c-aba0-871746c9a8b0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:12.094233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e080c8-784b-4b8c-aba0-871746c9a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:12.094333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e080c8-784b-4b8c-aba0-871746c9a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:12.153497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-45e080c8-784b-4b8c-aba0-871746c9a8b0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:12.155458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-45e080c8-784b-4b8c-aba0-871746c9a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:12.197074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:12.316009 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 828/1661] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:12.325600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e9678-bbbc-41f4-b0dc-ed1d1d09e61c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:12.325906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e9678-bbbc-41f4-b0dc-ed1d1d09e61c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:12.326194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e9678-bbbc-41f4-b0dc-ed1d1d09e61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:12.376380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ce9e9678-bbbc-41f4-b0dc-ed1d1d09e61c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:12.383912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ce9e9678-bbbc-41f4-b0dc-ed1d1d09e61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:12.407862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 834/1662] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:12.409790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e080c8-784b-4b8c-aba0-871746c9a8b0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:12.426542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e080c8-784b-4b8c-aba0-871746c9a8b0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:12.440084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:12.444075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:12.444075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:12.444075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:12.444075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:12.444075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:12.444075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:12.466984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 829/1663] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:01:12.468289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e9678-bbbc-41f4-b0dc-ed1d1d09e61c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:12.478748 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e9678-bbbc-41f4-b0dc-ed1d1d09e61c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:12.487779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4c71d-251f-40f7-b5af-fbe383d692be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:12.488471 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c71d-251f-40f7-b5af-fbe383d692be admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:12.488789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c71d-251f-40f7-b5af-fbe383d692be admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:12.489071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c71d-251f-40f7-b5af-fbe383d692be admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:12.489781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4c71d-251f-40f7-b5af-fbe383d692be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:12.493221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4c71d-251f-40f7-b5af-fbe383d692be admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:12.605706 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:12.608234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:12.608476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 835/1664] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:12 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:12.613402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4c71d-251f-40f7-b5af-fbe383d692be admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:12.715940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4c71d-251f-40f7-b5af-fbe383d692be admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:12.716215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c719e-bb85-45e1-a49e-b30119b9b82e admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:12.727702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd249d6-37dd-4531-80f6-6ae07c942db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:12.729187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd249d6-37dd-4531-80f6-6ae07c942db4 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:12.729689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd249d6-37dd-4531-80f6-6ae07c942db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:12.730195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd249d6-37dd-4531-80f6-6ae07c942db4 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:12.731139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd249d6-37dd-4531-80f6-6ae07c942db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:12.733950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd249d6-37dd-4531-80f6-6ae07c942db4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:12.809897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b920f4-78d5-4157-ba60-3726c5f7cf2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:12.810918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b920f4-78d5-4157-ba60-3726c5f7cf2b service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:12.811408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b920f4-78d5-4157-ba60-3726c5f7cf2b service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:12.811888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b920f4-78d5-4157-ba60-3726c5f7cf2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:12.814843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e6b920f4-78d5-4157-ba60-3726c5f7cf2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:12.864103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 836/1665] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:12 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:12.866142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62299590-f0a5-4c9a-99bd-a431dce2e5f3 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:12.866589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62299590-f0a5-4c9a-99bd-a431dce2e5f3 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:12.867054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62299590-f0a5-4c9a-99bd-a431dce2e5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:12.913522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b920f4-78d5-4157-ba60-3726c5f7cf2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:12.922087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-62299590-f0a5-4c9a-99bd-a431dce2e5f3 None None] MFA Rules not processed for user `55ed193c93c446a8b373f770512e5160`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:12.928380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-62299590-f0a5-4c9a-99bd-a431dce2e5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:12.941385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b920f4-78d5-4157-ba60-3726c5f7cf2b service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:12.965355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 830/1666] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:12.981267 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b920f4-78d5-4157-ba60-3726c5f7cf2b service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:13.023289 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b920f4-78d5-4157-ba60-3726c5f7cf2b service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:13.053699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587e9dd-3277-420d-b8f2-8fda783dff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:13.056991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587e9dd-3277-420d-b8f2-8fda783dff00 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:13.058842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587e9dd-3277-420d-b8f2-8fda783dff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:13.062895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587e9dd-3277-420d-b8f2-8fda783dff00 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:13.062895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587e9dd-3277-420d-b8f2-8fda783dff00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:13.067300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587e9dd-3277-420d-b8f2-8fda783dff00 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:13.102045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 831/1667] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:13.107659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587e9dd-3277-420d-b8f2-8fda783dff00 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:13.119899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 837/1668] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:13.161946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcef1b-1fff-4f3e-b2b2-3d953fd349cb None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:13.163647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587e9dd-3277-420d-b8f2-8fda783dff00 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:13.166737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcef1b-1fff-4f3e-b2b2-3d953fd349cb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:13.167466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcef1b-1fff-4f3e-b2b2-3d953fd349cb None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:13.170400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 838/1669] 10.222.0.17 () {54 vars in 962 bytes} [Sun Jul 24 20:01:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:13.177374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf694-0cd2-4770-bc99-c71742120de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:13.178078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf694-0cd2-4770-bc99-c71742120de3 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:13.180589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf694-0cd2-4770-bc99-c71742120de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:13.182735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf694-0cd2-4770-bc99-c71742120de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:13.183699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bf694-0cd2-4770-bc99-c71742120de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:13.188504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bf694-0cd2-4770-bc99-c71742120de3 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:13.189643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb23a86-f20e-4b7f-88b4-e6ccf4bef883 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:13.189643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb23a86-f20e-4b7f-88b4-e6ccf4bef883 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:13.189840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb23a86-f20e-4b7f-88b4-e6ccf4bef883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:13.213624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 832/1670] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:13.217926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf694-0cd2-4770-bc99-c71742120de3 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:13.229471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf694-0cd2-4770-bc99-c71742120de3 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:13.235489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8273bb-2eac-4f6b-8ae7-31573b672f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:13.236862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8273bb-2eac-4f6b-8ae7-31573b672f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:13.237318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8273bb-2eac-4f6b-8ae7-31573b672f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:13.237752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8273bb-2eac-4f6b-8ae7-31573b672f21 admin admin] PATH_INFO: `/v3/projects/78b17ea67fc94e549615dc8c0ce7ee24/users/21eed39015cd486db68951b432e6ab25/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:13.259660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8273bb-2eac-4f6b-8ae7-31573b672f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b17ea67fc94e549615dc8c0ce7ee24, user_id=21eed39015cd486db68951b432e6ab25, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:13.262855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8273bb-2eac-4f6b-8ae7-31573b672f21 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:13.273261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1eb23a86-f20e-4b7f-88b4-e6ccf4bef883 None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:13.280704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1eb23a86-f20e-4b7f-88b4-e6ccf4bef883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:13.382385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 833/1671] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:13 2022] PUT /identity/v3/projects/78b17ea67fc94e549615dc8c0ce7ee24/users/21eed39015cd486db68951b432e6ab25/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:13.385673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa5fbd-2869-4490-9860-d8bddffc6bed None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:13.389080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa5fbd-2869-4490-9860-d8bddffc6bed None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:13.389371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa5fbd-2869-4490-9860-d8bddffc6bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:13.478383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 839/1672] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Jul 24 20:01:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:13.479386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb23a86-f20e-4b7f-88b4-e6ccf4bef883 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:13.512019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb23a86-f20e-4b7f-88b4-e6ccf4bef883 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:13.519593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ddaa5fbd-2869-4490-9860-d8bddffc6bed None None] MFA Rules not processed for user `21eed39015cd486db68951b432e6ab25`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:13.530841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ddaa5fbd-2869-4490-9860-d8bddffc6bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:13.697987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9b673-8e70-4e04-85ac-eec2af063c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:13.700386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9b673-8e70-4e04-85ac-eec2af063c4f service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:13.700793 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9b673-8e70-4e04-85ac-eec2af063c4f service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:13.701148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9b673-8e70-4e04-85ac-eec2af063c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:13.720852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 834/1673] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:13.732657 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa5fbd-2869-4490-9860-d8bddffc6bed None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:13.735264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ddaa5fbd-2869-4490-9860-d8bddffc6bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:13.833277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9b673-8e70-4e04-85ac-eec2af063c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:13.835687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9b673-8e70-4e04-85ac-eec2af063c4f service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:13.857914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 840/1674] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:13.858892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9b673-8e70-4e04-85ac-eec2af063c4f service nova] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:13.991419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa5fbd-2869-4490-9860-d8bddffc6bed None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:13.991570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:14.024308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:14.031576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:14.031576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:14.031576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] PATH_INFO: `/v3/users/ed7495b443ce48dd94b690b7c363e93c` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:14.036853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9b673-8e70-4e04-85ac-eec2af063c4f service nova] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:14.045515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7495b443ce48dd94b690b7c363e93c)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:14.047804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:14.061190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d87b6-ad68-4b07-b760-071c12bd8e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:14.062928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d87b6-ad68-4b07-b760-071c12bd8e48 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:14.063491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d87b6-ad68-4b07-b760-071c12bd8e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:14.063845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d87b6-ad68-4b07-b760-071c12bd8e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:14.069450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d87b6-ad68-4b07-b760-071c12bd8e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:14.085331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d87b6-ad68-4b07-b760-071c12bd8e48 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:14.241754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 841/1675] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:13 2022] POST /identity/v3/projects => generated 371 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:14.246415 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d87b6-ad68-4b07-b760-071c12bd8e48 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:14.389203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7495b443ce48dd94b690b7c363e93c'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:14.428026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7495b443ce48dd94b690b7c363e93c'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:14.432011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7495b443ce48dd94b690b7c363e93c'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:14.498715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d87b6-ad68-4b07-b760-071c12bd8e48 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:14.515036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b3b1-8be2-498c-9a2d-7a441806725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:14.516096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b3b1-8be2-498c-9a2d-7a441806725c admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:14.516541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b3b1-8be2-498c-9a2d-7a441806725c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:14.517000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b3b1-8be2-498c-9a2d-7a441806725c admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:14.519378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b3b1-8be2-498c-9a2d-7a441806725c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:14.523852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b3b1-8be2-498c-9a2d-7a441806725c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:14.555337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7495b443ce48dd94b690b7c363e93c'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:14.621690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e4f3806e-4851-4e93-acd2-5ed2198b5f49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7495b443ce48dd94b690b7c363e93c'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:14.645981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 842/1676] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:14 2022] POST /identity/v3/users => generated 292 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:14.651190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b3b1-8be2-498c-9a2d-7a441806725c admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:14.655550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e2f4b3b1-8be2-498c-9a2d-7a441806725c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:14.681240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 835/1677] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:13 2022] DELETE /identity/v3/users/ed7495b443ce48dd94b690b7c363e93c => generated 0 bytes in 960 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:14.694236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe3788-1436-4223-9b8a-1436237571f5 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:14.698814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe3788-1436-4223-9b8a-1436237571f5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:14.699129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe3788-1436-4223-9b8a-1436237571f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:14.726130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b3b1-8be2-498c-9a2d-7a441806725c admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:14.774343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-0efe3788-1436-4223-9b8a-1436237571f5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:14.775829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0efe3788-1436-4223-9b8a-1436237571f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:14.816116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:14.908022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba04c7-a751-4058-81e5-47c0d4dbbd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:14.908824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba04c7-a751-4058-81e5-47c0d4dbbd49 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:14.909439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba04c7-a751-4058-81e5-47c0d4dbbd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:14.909798 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba04c7-a751-4058-81e5-47c0d4dbbd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:14.910567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba04c7-a751-4058-81e5-47c0d4dbbd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:14.912755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba04c7-a751-4058-81e5-47c0d4dbbd49 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:14.926118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 843/1678] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:14.932289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba04c7-a751-4058-81e5-47c0d4dbbd49 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:14.952049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba04c7-a751-4058-81e5-47c0d4dbbd49 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:14.957918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27e59c-9b83-453d-a823-586469a289aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:14.958849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27e59c-9b83-453d-a823-586469a289aa admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:14.959159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27e59c-9b83-453d-a823-586469a289aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:14.959451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27e59c-9b83-453d-a823-586469a289aa admin admin] PATH_INFO: `/v3/projects/2f8690f9245c41119cbcb3ba00544d0f/users/3a316e25466c47a5b2f7a80b1ae24994/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:14.999331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 836/1679] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:15.007410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27e59c-9b83-453d-a823-586469a289aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8690f9245c41119cbcb3ba00544d0f, user_id=3a316e25466c47a5b2f7a80b1ae24994, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:15.010775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27e59c-9b83-453d-a823-586469a289aa admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:15.030917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe3788-1436-4223-9b8a-1436237571f5 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:15.032364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0efe3788-1436-4223-9b8a-1436237571f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:15.111829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 844/1680] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:14 2022] PUT /identity/v3/projects/2f8690f9245c41119cbcb3ba00544d0f/users/3a316e25466c47a5b2f7a80b1ae24994/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:15.113050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27e59c-9b83-453d-a823-586469a289aa admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:15.126401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe3788-1436-4223-9b8a-1436237571f5 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:15.204907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27e59c-9b83-453d-a823-586469a289aa admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:15.216070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ae427-dd76-424e-9dcd-e1f576d17fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:15.217008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ae427-dd76-424e-9dcd-e1f576d17fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:15.217395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ae427-dd76-424e-9dcd-e1f576d17fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:15.217844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ae427-dd76-424e-9dcd-e1f576d17fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:15.218925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ae427-dd76-424e-9dcd-e1f576d17fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:15.221576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ae427-dd76-424e-9dcd-e1f576d17fd8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:15.243839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 845/1681] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:15.244947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ae427-dd76-424e-9dcd-e1f576d17fd8 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:15.245273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8c23b-3ae1-4c2b-83df-aa3c9d19d8ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:15.245965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c23b-3ae1-4c2b-83df-aa3c9d19d8ff service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:15.246255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c23b-3ae1-4c2b-83df-aa3c9d19d8ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:15.246539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c23b-3ae1-4c2b-83df-aa3c9d19d8ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:15.265608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8c23b-3ae1-4c2b-83df-aa3c9d19d8ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:15.267118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8c23b-3ae1-4c2b-83df-aa3c9d19d8ff service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:15.267685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ae427-dd76-424e-9dcd-e1f576d17fd8 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:15.283225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3f39a-e5eb-4ea0-8ed5-952a2ea5e3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:15.283225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3f39a-e5eb-4ea0-8ed5-952a2ea5e3db admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:15.283225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3f39a-e5eb-4ea0-8ed5-952a2ea5e3db admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:15.283225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3f39a-e5eb-4ea0-8ed5-952a2ea5e3db admin admin] PATH_INFO: `/v3/projects/2f8690f9245c41119cbcb3ba00544d0f/users/3a316e25466c47a5b2f7a80b1ae24994/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:15.288250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3f39a-e5eb-4ea0-8ed5-952a2ea5e3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8690f9245c41119cbcb3ba00544d0f, user_id=3a316e25466c47a5b2f7a80b1ae24994, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:15.294117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3f39a-e5eb-4ea0-8ed5-952a2ea5e3db admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:15.299631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 837/1682] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:15.322186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 846/1683] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:15 2022] PUT /identity/v3/projects/2f8690f9245c41119cbcb3ba00544d0f/users/3a316e25466c47a5b2f7a80b1ae24994/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:15.324692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b099-e445-4f0e-a3d3-33dbf5f1ec6f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:15.325105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b099-e445-4f0e-a3d3-33dbf5f1ec6f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:15.325614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b099-e445-4f0e-a3d3-33dbf5f1ec6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:15.376303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e596b099-e445-4f0e-a3d3-33dbf5f1ec6f None None] MFA Rules not processed for user `3a316e25466c47a5b2f7a80b1ae24994`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:15.377847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e596b099-e445-4f0e-a3d3-33dbf5f1ec6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:15.519685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 838/1684] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:15 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:15.526897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77f613-943a-4cf5-b1d7-658e8e99b742 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:15.526897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77f613-943a-4cf5-b1d7-658e8e99b742 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:15.539452 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77f613-943a-4cf5-b1d7-658e8e99b742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:15.548596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b099-e445-4f0e-a3d3-33dbf5f1ec6f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:15.550228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e596b099-e445-4f0e-a3d3-33dbf5f1ec6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:15.592138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-fc77f613-943a-4cf5-b1d7-658e8e99b742 None None] MFA Rules not processed for user `0602248c62664dd4a8809225d14ca5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:15.593495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fc77f613-943a-4cf5-b1d7-658e8e99b742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:15.658985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b099-e445-4f0e-a3d3-33dbf5f1ec6f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:15.663471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1022272-acf6-45c8-9c30-5a1e385a3bc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:15.665625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1022272-acf6-45c8-9c30-5a1e385a3bc4 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:15.666369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1022272-acf6-45c8-9c30-5a1e385a3bc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:15.666369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1022272-acf6-45c8-9c30-5a1e385a3bc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:15.681197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1022272-acf6-45c8-9c30-5a1e385a3bc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:15.686482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1022272-acf6-45c8-9c30-5a1e385a3bc4 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:15.715499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 839/1685] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:15 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:15.727108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:15.730177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 847/1686] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:15.741266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebf2d9-07d6-48ec-92c0-b6b5164df1d1 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:15.741610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebf2d9-07d6-48ec-92c0-b6b5164df1d1 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:15.741981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebf2d9-07d6-48ec-92c0-b6b5164df1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:15.768045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a1ebf2d9-07d6-48ec-92c0-b6b5164df1d1 None None] MFA Rules not processed for user `0602248c62664dd4a8809225d14ca5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:15.769765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a1ebf2d9-07d6-48ec-92c0-b6b5164df1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:15.795416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 840/1687] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:15.807363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77f613-943a-4cf5-b1d7-658e8e99b742 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:15.810592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fc77f613-943a-4cf5-b1d7-658e8e99b742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:16.029730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77f613-943a-4cf5-b1d7-658e8e99b742 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:16.036925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea20073-19b0-406d-88f1-3c7a5d3d882d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:16.044758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea20073-19b0-406d-88f1-3c7a5d3d882d service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:16.049260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea20073-19b0-406d-88f1-3c7a5d3d882d service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:16.049260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea20073-19b0-406d-88f1-3c7a5d3d882d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:16.057087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea20073-19b0-406d-88f1-3c7a5d3d882d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:16.058942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea20073-19b0-406d-88f1-3c7a5d3d882d service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:16.077713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 848/1688] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:16.282008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebf2d9-07d6-48ec-92c0-b6b5164df1d1 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:16.283853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a1ebf2d9-07d6-48ec-92c0-b6b5164df1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:16.539527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebf2d9-07d6-48ec-92c0-b6b5164df1d1 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:16.547498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:16.548414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:16.550481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:16.550760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] PATH_INFO: `/v3/projects/60402d6f6eec43b1b215a506135c7496` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:16.553450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60402d6f6eec43b1b215a506135c7496)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:16.555286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: WARNING py.warnings [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 20:01:16.555286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: warnings.warn(msg) Jul 24 20:01:16.555286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: Jul 24 20:01:16.556482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:16.716923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60402d6f6eec43b1b215a506135c7496', 'request_id': 'req-f751bb18-23c3-49a4-9090-ddf6cddece98'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:16.733136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60402d6f6eec43b1b215a506135c7496', 'request_id': 'req-f751bb18-23c3-49a4-9090-ddf6cddece98'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:16.814357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60402d6f6eec43b1b215a506135c7496', 'request_id': 'req-f751bb18-23c3-49a4-9090-ddf6cddece98'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:16.816038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5626b-9c9b-4675-8cae-e641ecf805bc None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:16.818955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5626b-9c9b-4675-8cae-e641ecf805bc None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:16.818955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5626b-9c9b-4675-8cae-e641ecf805bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:16.865938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] The token cache is being invalidate because project 60402d6f6eec43b1b215a506135c7496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:16.866634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:16.869797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:16.876560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1cc5626b-9c9b-4675-8cae-e641ecf805bc None None] MFA Rules not processed for user `5221431e0552435d9ff82f8ce75b624e`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:16.887020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1cc5626b-9c9b-4675-8cae-e641ecf805bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:16.890854 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 841/1689] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:16 2022] DELETE /identity/v3/projects/60402d6f6eec43b1b215a506135c7496 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:16.892580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:16.895951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:16.899098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:17.104994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751bb18-23c3-49a4-9090-ddf6cddece98 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:17.106684 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 849/1690] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:17.120327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fb011-8837-4b9e-8eb5-4c375db813a5 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:17.120327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fb011-8837-4b9e-8eb5-4c375db813a5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:17.122445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fb011-8837-4b9e-8eb5-4c375db813a5 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:17.127895 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 850/1691] 10.222.0.17 () {54 vars in 912 bytes} [Sun Jul 24 20:01:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:17.131572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:17.132038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35eb42-76f8-4bea-8053-95d56deb2981 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:17.133834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:17.134369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35eb42-76f8-4bea-8053-95d56deb2981 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:17.134877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35eb42-76f8-4bea-8053-95d56deb2981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:17.135371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:17.137067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] PATH_INFO: `/v3/users/0b8877be001d4db784d9ae8c1d4fed0f` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:17.138935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8877be001d4db784d9ae8c1d4fed0f)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:17.142937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:17.191959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ad35eb42-76f8-4bea-8053-95d56deb2981 None None] MFA Rules not processed for user `5221431e0552435d9ff82f8ce75b624e`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:17.192344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ad35eb42-76f8-4bea-8053-95d56deb2981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:17.216287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 851/1692] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:17.218414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f0f83-d785-4395-a936-d12dfda52ed8 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:17.220463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f0f83-d785-4395-a936-d12dfda52ed8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:17.220463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f0f83-d785-4395-a936-d12dfda52ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:17.323501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e59f0f83-d785-4395-a936-d12dfda52ed8 None None] MFA Rules not processed for user `b9763f9899214df3929f3816f009e857`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:17.333723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e59f0f83-d785-4395-a936-d12dfda52ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:17.501656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8877be001d4db784d9ae8c1d4fed0f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:17.523148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8877be001d4db784d9ae8c1d4fed0f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:17.524493 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8877be001d4db784d9ae8c1d4fed0f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:17.540627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 852/1693] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:01:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:17.542162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f0f83-d785-4395-a936-d12dfda52ed8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:17.544412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e59f0f83-d785-4395-a936-d12dfda52ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:17.622213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8877be001d4db784d9ae8c1d4fed0f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:17.638539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:17.661891 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8877be001d4db784d9ae8c1d4fed0f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:17.681075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 842/1694] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:16 2022] DELETE /identity/v3/users/0b8877be001d4db784d9ae8c1d4fed0f => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:17.682664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:17.684916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:17.738145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f0f83-d785-4395-a936-d12dfda52ed8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:17.845031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f441ed-b6d5-403a-a949-1233f78397c9 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:17.853021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:17.854650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:17.855130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:17.855469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:17.855950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:17.856369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:17.856675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:17.856970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] PATH_INFO: `/v3/users/21eed39015cd486db68951b432e6ab25` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:17.859206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:17.862013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21eed39015cd486db68951b432e6ab25)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:17.864096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:17.984560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:17.987642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:18.004218 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 853/1695] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:18.005251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:18.007019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:18.154519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21eed39015cd486db68951b432e6ab25'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:18.171543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21eed39015cd486db68951b432e6ab25'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:18.172944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21eed39015cd486db68951b432e6ab25'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:18.182227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedd69c-628b-48da-a993-50c99db04133 service nova] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:18.190457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fda302-7924-4e1f-b978-2432393ce8d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:18.191337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fda302-7924-4e1f-b978-2432393ce8d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:18.191730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fda302-7924-4e1f-b978-2432393ce8d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:18.192362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fda302-7924-4e1f-b978-2432393ce8d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:18.200215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-04fda302-7924-4e1f-b978-2432393ce8d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:18.208816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:18.222302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fda302-7924-4e1f-b978-2432393ce8d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:18.222302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fda302-7924-4e1f-b978-2432393ce8d9 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:18.246171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 854/1696] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:18.251896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88546c20-70f8-448d-8db6-0d6339bba154 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:18.251896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88546c20-70f8-448d-8db6-0d6339bba154 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:18.251896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88546c20-70f8-448d-8db6-0d6339bba154 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:18.255184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 855/1697] 10.222.0.17 () {54 vars in 912 bytes} [Sun Jul 24 20:01:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:18.274946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edb24d-0222-44c7-8b07-842285864b69 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:18.275245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edb24d-0222-44c7-8b07-842285864b69 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:18.275525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edb24d-0222-44c7-8b07-842285864b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:18.288135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21eed39015cd486db68951b432e6ab25'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:18.311814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cb093ecf-6acb-449c-a56b-6efc38cff824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21eed39015cd486db68951b432e6ab25'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:18.329867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 843/1698] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:17 2022] DELETE /identity/v3/users/21eed39015cd486db68951b432e6ab25 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:18.330012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-05edb24d-0222-44c7-8b07-842285864b69 None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:18.330012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-05edb24d-0222-44c7-8b07-842285864b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:18.330151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a40266-bece-47fe-98c1-a49ec578858c None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:18.330151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a40266-bece-47fe-98c1-a49ec578858c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:18.330151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a40266-bece-47fe-98c1-a49ec578858c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:18.373903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-06a40266-bece-47fe-98c1-a49ec578858c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:18.378510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-06a40266-bece-47fe-98c1-a49ec578858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:18.568025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 856/1699] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:01:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:18.587607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edb24d-0222-44c7-8b07-842285864b69 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:18.589100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-05edb24d-0222-44c7-8b07-842285864b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:18.597303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 844/1700] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:18.656728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a40266-bece-47fe-98c1-a49ec578858c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:18.662080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-06a40266-bece-47fe-98c1-a49ec578858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:18.795009 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a40266-bece-47fe-98c1-a49ec578858c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:18.797005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141367dd-6815-493f-a988-2c0606a09672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:18.797978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141367dd-6815-493f-a988-2c0606a09672 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:18.798264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141367dd-6815-493f-a988-2c0606a09672 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:18.798543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141367dd-6815-493f-a988-2c0606a09672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:18.812978 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edb24d-0222-44c7-8b07-842285864b69 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:18.813953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141367dd-6815-493f-a988-2c0606a09672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:18.815726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141367dd-6815-493f-a988-2c0606a09672 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:18.828660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6171d-98d9-46a7-b498-e62863401c32 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca33ef49ba44832a7fb45b2317d211c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:18.829550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6171d-98d9-46a7-b498-e62863401c32 service glance] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:18.829850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6171d-98d9-46a7-b498-e62863401c32 service glance] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:18.830144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6171d-98d9-46a7-b498-e62863401c32 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:18.854397 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6171d-98d9-46a7-b498-e62863401c32 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:18.855877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6171d-98d9-46a7-b498-e62863401c32 service glance] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:18.861305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 845/1701] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:01:18.883104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 857/1702] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:01:19.133881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:19.286972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141367dd-6815-493f-a988-2c0606a09672 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:19.289534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-141367dd-6815-493f-a988-2c0606a09672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:19.312766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141367dd-6815-493f-a988-2c0606a09672 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:19.348861 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:19.356115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:19.356723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:19.357204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] PATH_INFO: `/v3/projects/bd9a62a6eaaa48a288c6d76bc368f0ba` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:19.360403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd9a62a6eaaa48a288c6d76bc368f0ba)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:19.369390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:19.512020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732e22e-0a7b-49d4-9517-16e77301cc1d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:19.512380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732e22e-0a7b-49d4-9517-16e77301cc1d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:19.512756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732e22e-0a7b-49d4-9517-16e77301cc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:19.560337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e732e22e-0a7b-49d4-9517-16e77301cc1d None None] MFA Rules not processed for user `55ed193c93c446a8b373f770512e5160`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:19.561585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e732e22e-0a7b-49d4-9517-16e77301cc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:19.673701 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 858/1703] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:19.675487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad7fb9-0e82-4bc7-844d-f8533518b21b None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:19.675794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad7fb9-0e82-4bc7-844d-f8533518b21b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:19.676076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad7fb9-0e82-4bc7-844d-f8533518b21b None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:19.677849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 859/1704] 10.222.0.17 () {54 vars in 961 bytes} [Sun Jul 24 20:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:19.684078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6654adf-321d-4aee-b669-108ea22a5509 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:19.684378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6654adf-321d-4aee-b669-108ea22a5509 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:19.684731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6654adf-321d-4aee-b669-108ea22a5509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:19.716457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f6654adf-321d-4aee-b669-108ea22a5509 None None] MFA Rules not processed for user `1c1b23b83544423abc6919d805916b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:19.717772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f6654adf-321d-4aee-b669-108ea22a5509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:19.815069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 860/1705] 10.222.0.17 () {58 vars in 1058 bytes} [Sun Jul 24 20:01:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:19.815656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd9a62a6eaaa48a288c6d76bc368f0ba', 'request_id': 'req-0f95b563-585d-447d-bd98-acd244e308ed'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:19.817250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61bdf-e5ec-49a6-83de-a61381f35d68 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:19.818400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61bdf-e5ec-49a6-83de-a61381f35d68 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:19.818794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61bdf-e5ec-49a6-83de-a61381f35d68 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:19.821568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 861/1706] 10.222.0.17 () {54 vars in 912 bytes} [Sun Jul 24 20:01:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:19.822732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed61bdf-e5ec-49a6-83de-a61381f35d68 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:19.825873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd9a62a6eaaa48a288c6d76bc368f0ba', 'request_id': 'req-0f95b563-585d-447d-bd98-acd244e308ed'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:19.940135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd9a62a6eaaa48a288c6d76bc368f0ba', 'request_id': 'req-0f95b563-585d-447d-bd98-acd244e308ed'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:19.957207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed61bdf-e5ec-49a6-83de-a61381f35d68 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:19.964741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] The token cache is being invalidate because project bd9a62a6eaaa48a288c6d76bc368f0ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:19.965337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:19.966314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f95b563-585d-447d-bd98-acd244e308ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:19.969251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 846/1707] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:19 2022] DELETE /identity/v3/projects/bd9a62a6eaaa48a288c6d76bc368f0ba => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:19.973235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5d777-96b3-45e2-927c-91cb02f0e7bf None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:19.973890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adafd4c-2bab-4cd9-ad58-48aded5dbdf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:19.974152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5d777-96b3-45e2-927c-91cb02f0e7bf None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:19.974586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adafd4c-2bab-4cd9-ad58-48aded5dbdf9 service placement] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:19.974884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adafd4c-2bab-4cd9-ad58-48aded5dbdf9 service placement] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:19.975146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adafd4c-2bab-4cd9-ad58-48aded5dbdf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:19.975285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5d777-96b3-45e2-927c-91cb02f0e7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:19.979501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6adafd4c-2bab-4cd9-ad58-48aded5dbdf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:20.015525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adafd4c-2bab-4cd9-ad58-48aded5dbdf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:20.019044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-70b5d777-96b3-45e2-927c-91cb02f0e7bf None None] MFA Rules not processed for user `b9763f9899214df3929f3816f009e857`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:20.021228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-70b5d777-96b3-45e2-927c-91cb02f0e7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:20.023451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:20.026655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adafd4c-2bab-4cd9-ad58-48aded5dbdf9 service placement] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:20.089199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 862/1708] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:20.090858 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2adea-8a21-4b0b-923a-bf2a5b72f80d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:20.091154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2adea-8a21-4b0b-923a-bf2a5b72f80d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:20.091435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2adea-8a21-4b0b-923a-bf2a5b72f80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:20.127958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 847/1709] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:01:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:20.134023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5d777-96b3-45e2-927c-91cb02f0e7bf None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:20.137449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-70b5d777-96b3-45e2-927c-91cb02f0e7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:20.168072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-40f2adea-8a21-4b0b-923a-bf2a5b72f80d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:20.169572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-40f2adea-8a21-4b0b-923a-bf2a5b72f80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:20.292336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5d777-96b3-45e2-927c-91cb02f0e7bf None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:20.309851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:20.311232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:20.311614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:20.312005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] PATH_INFO: `/v3/projects/78b17ea67fc94e549615dc8c0ce7ee24` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:20.323570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=78b17ea67fc94e549615dc8c0ce7ee24)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:20.323570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:20.363621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 863/1710] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:20.364678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2adea-8a21-4b0b-923a-bf2a5b72f80d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:20.380213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-40f2adea-8a21-4b0b-923a-bf2a5b72f80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:20.398815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2adea-8a21-4b0b-923a-bf2a5b72f80d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:20.414166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0c271-df26-44e4-8e18-055937d33a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:20.415561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0c271-df26-44e4-8e18-055937d33a96 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:20.415860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0c271-df26-44e4-8e18-055937d33a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:20.416142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0c271-df26-44e4-8e18-055937d33a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:20.467537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0c271-df26-44e4-8e18-055937d33a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:20.473150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0c271-df26-44e4-8e18-055937d33a96 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:20.480924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b17ea67fc94e549615dc8c0ce7ee24', 'request_id': 'req-e332019d-c3f6-4ca0-97cf-33db8f404b4d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:20.486842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 864/1711] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:20.488170 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5100752-b915-4238-8e47-ac3bdc55b352 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:20.488353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5100752-b915-4238-8e47-ac3bdc55b352 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:20.494456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5100752-b915-4238-8e47-ac3bdc55b352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:20.497052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b17ea67fc94e549615dc8c0ce7ee24', 'request_id': 'req-e332019d-c3f6-4ca0-97cf-33db8f404b4d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:20.537139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d5100752-b915-4238-8e47-ac3bdc55b352 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:20.538762 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d5100752-b915-4238-8e47-ac3bdc55b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:20.575471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b17ea67fc94e549615dc8c0ce7ee24', 'request_id': 'req-e332019d-c3f6-4ca0-97cf-33db8f404b4d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:20.596185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] The token cache is being invalidate because project 78b17ea67fc94e549615dc8c0ce7ee24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:20.599139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:20.609577 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:20.612314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 848/1712] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:20 2022] DELETE /identity/v3/projects/78b17ea67fc94e549615dc8c0ce7ee24 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:20.617279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:20.619456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:20.655589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 865/1713] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:20.660856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5100752-b915-4238-8e47-ac3bdc55b352 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:20.676867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5100752-b915-4238-8e47-ac3bdc55b352 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:20.682879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77470e62-5383-4dd1-9d8d-41922af2cc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:20.684047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77470e62-5383-4dd1-9d8d-41922af2cc3a admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:20.684339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77470e62-5383-4dd1-9d8d-41922af2cc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:20.684621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77470e62-5383-4dd1-9d8d-41922af2cc3a admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:20.685686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77470e62-5383-4dd1-9d8d-41922af2cc3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:20.687581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77470e62-5383-4dd1-9d8d-41922af2cc3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:20.688770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77470e62-5383-4dd1-9d8d-41922af2cc3a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:20.721925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 866/1714] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:01:20.723185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77470e62-5383-4dd1-9d8d-41922af2cc3a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:20.740324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77470e62-5383-4dd1-9d8d-41922af2cc3a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:20.769703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332019d-c3f6-4ca0-97cf-33db8f404b4d admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:20.776415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:20.779342 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:20.779342 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:20.779342 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] PATH_INFO: `/v3/users/3a316e25466c47a5b2f7a80b1ae24994` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:20.782421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a316e25466c47a5b2f7a80b1ae24994)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:20.786362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:20.788742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3e706-a566-4d4f-9d26-c90e6775bd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:20.789432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3e706-a566-4d4f-9d26-c90e6775bd37 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:20.789750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3e706-a566-4d4f-9d26-c90e6775bd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:20.790447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3e706-a566-4d4f-9d26-c90e6775bd37 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:20.791306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3e706-a566-4d4f-9d26-c90e6775bd37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:20.793570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3e706-a566-4d4f-9d26-c90e6775bd37 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:20.931872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 867/1715] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:20 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:20.933032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3e706-a566-4d4f-9d26-c90e6775bd37 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:21.028554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3e706-a566-4d4f-9d26-c90e6775bd37 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:21.035526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db70bd2-28b0-4749-90e1-8a60516c7fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:21.037525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db70bd2-28b0-4749-90e1-8a60516c7fbb admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:21.038051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db70bd2-28b0-4749-90e1-8a60516c7fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:21.042717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db70bd2-28b0-4749-90e1-8a60516c7fbb admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:21.042717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db70bd2-28b0-4749-90e1-8a60516c7fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:21.042717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db70bd2-28b0-4749-90e1-8a60516c7fbb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:21.091165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a316e25466c47a5b2f7a80b1ae24994'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:21.103900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a316e25466c47a5b2f7a80b1ae24994'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:21.104721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a316e25466c47a5b2f7a80b1ae24994'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:21.146541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 868/1716] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:20 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:21.156030 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db70bd2-28b0-4749-90e1-8a60516c7fbb admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:21.204439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:21.229193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db70bd2-28b0-4749-90e1-8a60516c7fbb admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:21.232870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a316e25466c47a5b2f7a80b1ae24994'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:21.258015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223e728-529d-4d68-8a62-5290a79319f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:21.270708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223e728-529d-4d68-8a62-5290a79319f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:21.272546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223e728-529d-4d68-8a62-5290a79319f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:21.275527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223e728-529d-4d68-8a62-5290a79319f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:21.280361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223e728-529d-4d68-8a62-5290a79319f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:21.282135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223e728-529d-4d68-8a62-5290a79319f1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:21.297529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a316e25466c47a5b2f7a80b1ae24994'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:21.330274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 849/1717] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:20 2022] DELETE /identity/v3/users/3a316e25466c47a5b2f7a80b1ae24994 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:21.367209 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 869/1718] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:21.368019 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:21.369647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:21.580361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a63d91-d2c5-4f43-bd35-83a5e1737767 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:21.615484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f1299-dcba-419c-a7a9-faaaa1ba72bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:21.616232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1299-dcba-419c-a7a9-faaaa1ba72bf admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:21.616963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1299-dcba-419c-a7a9-faaaa1ba72bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:21.616963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1299-dcba-419c-a7a9-faaaa1ba72bf admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:21.617725 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f1299-dcba-419c-a7a9-faaaa1ba72bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:21.619498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f1299-dcba-419c-a7a9-faaaa1ba72bf admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:21.649571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 850/1719] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:21.650321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223e728-529d-4d68-8a62-5290a79319f1 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:21.678055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223e728-529d-4d68-8a62-5290a79319f1 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:21.689899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b6c99-c547-4154-9c47-b7741f0e25d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:21.690927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b6c99-c547-4154-9c47-b7741f0e25d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:21.691288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b6c99-c547-4154-9c47-b7741f0e25d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:21.691643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b6c99-c547-4154-9c47-b7741f0e25d2 admin admin] PATH_INFO: `/v3/projects/e04b9dac428e438bab45a825c81d441d/users/0b85604bb02744d283ab904e8dea2d29/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:21.736439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b6c99-c547-4154-9c47-b7741f0e25d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04b9dac428e438bab45a825c81d441d, user_id=0b85604bb02744d283ab904e8dea2d29, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:21.745680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b6c99-c547-4154-9c47-b7741f0e25d2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:21.848757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 870/1720] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:21 2022] PUT /identity/v3/projects/e04b9dac428e438bab45a825c81d441d/users/0b85604bb02744d283ab904e8dea2d29/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:21.864902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc098a26-aa33-4aff-bd87-617d9e94535b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:21.871873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc098a26-aa33-4aff-bd87-617d9e94535b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:21.871873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc098a26-aa33-4aff-bd87-617d9e94535b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:21.927572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:22.014466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-fc098a26-aa33-4aff-bd87-617d9e94535b None None] MFA Rules not processed for user `0b85604bb02744d283ab904e8dea2d29`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:22.016426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fc098a26-aa33-4aff-bd87-617d9e94535b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:22.202618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 851/1721] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:22.203161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b6c99-c547-4154-9c47-b7741f0e25d2 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:22.263735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:22.283479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c35e0-14ff-4bd1-aeef-3887a6ce81b2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:22.284107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c35e0-14ff-4bd1-aeef-3887a6ce81b2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:22.284718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c35e0-14ff-4bd1-aeef-3887a6ce81b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:22.318562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a63c35e0-14ff-4bd1-aeef-3887a6ce81b2 None None] MFA Rules not processed for user `5221431e0552435d9ff82f8ce75b624e`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:22.320840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a63c35e0-14ff-4bd1-aeef-3887a6ce81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:22.332250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b6c99-c547-4154-9c47-b7741f0e25d2 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:22.338701 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39713970-1031-47b6-8729-02ded299c758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:22.339564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39713970-1031-47b6-8729-02ded299c758 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:22.339924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39713970-1031-47b6-8729-02ded299c758 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:22.340278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39713970-1031-47b6-8729-02ded299c758 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:22.341246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39713970-1031-47b6-8729-02ded299c758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:22.344069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39713970-1031-47b6-8729-02ded299c758 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:22.458841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 852/1722] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:22.466280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c35e0-14ff-4bd1-aeef-3887a6ce81b2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:22.475041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a63c35e0-14ff-4bd1-aeef-3887a6ce81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:22.494617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 871/1723] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:22 2022] POST /identity/v3/projects => generated 381 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:22.497940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39713970-1031-47b6-8729-02ded299c758 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:22.515306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c35e0-14ff-4bd1-aeef-3887a6ce81b2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:22.637325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:22.652362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39713970-1031-47b6-8729-02ded299c758 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:22.666895 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48355e5-8918-4f5b-a7fe-e59f614f9caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:22.668079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48355e5-8918-4f5b-a7fe-e59f614f9caa admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:22.668576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48355e5-8918-4f5b-a7fe-e59f614f9caa admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:22.669193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48355e5-8918-4f5b-a7fe-e59f614f9caa admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:22.670111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48355e5-8918-4f5b-a7fe-e59f614f9caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:22.673741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48355e5-8918-4f5b-a7fe-e59f614f9caa admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:22.697932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:22.698980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:22.699265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:22.699539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] PATH_INFO: `/v3/projects/2f8690f9245c41119cbcb3ba00544d0f` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:22.700832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8690f9245c41119cbcb3ba00544d0f)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:22.705065 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:22.823484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 872/1724] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:22 2022] POST /identity/v3/users => generated 297 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:22.826638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48355e5-8918-4f5b-a7fe-e59f614f9caa admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:22.854013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48355e5-8918-4f5b-a7fe-e59f614f9caa admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:22.859184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfaa26-444d-4a21-b768-bfaa2a86ae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:22.860349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfaa26-444d-4a21-b768-bfaa2a86ae79 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:22.860884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfaa26-444d-4a21-b768-bfaa2a86ae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:22.861836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfaa26-444d-4a21-b768-bfaa2a86ae79 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:22.862795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfaa26-444d-4a21-b768-bfaa2a86ae79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:22.864735 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfaa26-444d-4a21-b768-bfaa2a86ae79 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:22.879711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 873/1725] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:22.897127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfaa26-444d-4a21-b768-bfaa2a86ae79 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:22.916348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfaa26-444d-4a21-b768-bfaa2a86ae79 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:22.927212 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcbf42-88cb-4235-bcfe-696affffd068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:22.928529 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcbf42-88cb-4235-bcfe-696affffd068 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:22.929022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcbf42-88cb-4235-bcfe-696affffd068 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:22.930746 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcbf42-88cb-4235-bcfe-696affffd068 admin admin] PATH_INFO: `/v3/projects/71b96e36b4f8424a982bb50ba0d0ddb4/users/3e8bb4b1562b4f9f8013939e050033fd/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.017369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8690f9245c41119cbcb3ba00544d0f', 'request_id': 'req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:23.027278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcbf42-88cb-4235-bcfe-696affffd068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b96e36b4f8424a982bb50ba0d0ddb4, user_id=3e8bb4b1562b4f9f8013939e050033fd, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:23.046208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcbf42-88cb-4235-bcfe-696affffd068 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:23.051675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8690f9245c41119cbcb3ba00544d0f', 'request_id': 'req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:23.147132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8690f9245c41119cbcb3ba00544d0f', 'request_id': 'req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:23.162241 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 874/1726] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:22 2022] PUT /identity/v3/projects/71b96e36b4f8424a982bb50ba0d0ddb4/users/3e8bb4b1562b4f9f8013939e050033fd/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:23.167459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcbf42-88cb-4235-bcfe-696affffd068 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:23.173147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] The token cache is being invalidate because project 2f8690f9245c41119cbcb3ba00544d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:23.173529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:23.174850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-272dc216-3b90-4dfa-b08b-7d5f5a80c12a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:23.178437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 853/1727] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:22 2022] DELETE /identity/v3/projects/2f8690f9245c41119cbcb3ba00544d0f => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:23.183178 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe52d5-d8cc-450e-883c-cfc15b8a1b94 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.183474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe52d5-d8cc-450e-883c-cfc15b8a1b94 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.183802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe52d5-d8cc-450e-883c-cfc15b8a1b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.233853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-7bfe52d5-d8cc-450e-883c-cfc15b8a1b94 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:23.235158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7bfe52d5-d8cc-450e-883c-cfc15b8a1b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:23.316486 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcbf42-88cb-4235-bcfe-696affffd068 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:23.329970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ddefc-4bec-402e-a170-394296a40ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:23.330891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ddefc-4bec-402e-a170-394296a40ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.334220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ddefc-4bec-402e-a170-394296a40ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.334220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ddefc-4bec-402e-a170-394296a40ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.334598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ddefc-4bec-402e-a170-394296a40ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:23.338999 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ddefc-4bec-402e-a170-394296a40ad6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:23.358217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 875/1728] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:23.363425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ddefc-4bec-402e-a170-394296a40ad6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:23.367589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-022ddefc-4bec-402e-a170-394296a40ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:23.383070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 854/1729] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:23.391003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e180e3-0e72-49d1-9930-e9fd54c6d1c6 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.392774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e180e3-0e72-49d1-9930-e9fd54c6d1c6 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.392774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e180e3-0e72-49d1-9930-e9fd54c6d1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.408353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ddefc-4bec-402e-a170-394296a40ad6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:23.427061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1a9c3-14af-400e-93ad-b731038b1715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:23.428017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1a9c3-14af-400e-93ad-b731038b1715 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.428541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1a9c3-14af-400e-93ad-b731038b1715 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.428884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1a9c3-14af-400e-93ad-b731038b1715 admin admin] PATH_INFO: `/v3/projects/71b96e36b4f8424a982bb50ba0d0ddb4/users/3e8bb4b1562b4f9f8013939e050033fd/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.436747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1a9c3-14af-400e-93ad-b731038b1715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b96e36b4f8424a982bb50ba0d0ddb4, user_id=3e8bb4b1562b4f9f8013939e050033fd, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:23.436747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1a9c3-14af-400e-93ad-b731038b1715 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:23.447471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d9e180e3-0e72-49d1-9930-e9fd54c6d1c6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:23.448963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d9e180e3-0e72-49d1-9930-e9fd54c6d1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:23.459046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 855/1730] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:23.473723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e180e3-0e72-49d1-9930-e9fd54c6d1c6 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:23.479199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 876/1731] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:23 2022] PUT /identity/v3/projects/71b96e36b4f8424a982bb50ba0d0ddb4/users/3e8bb4b1562b4f9f8013939e050033fd/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:23.486926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e180e3-0e72-49d1-9930-e9fd54c6d1c6 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:23.489941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927d0a-9897-44c2-8d5c-6e28dd99f65f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.490283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927d0a-9897-44c2-8d5c-6e28dd99f65f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.490698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927d0a-9897-44c2-8d5c-6e28dd99f65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.521869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-09927d0a-9897-44c2-8d5c-6e28dd99f65f None None] MFA Rules not processed for user `3e8bb4b1562b4f9f8013939e050033fd`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:23.523447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-09927d0a-9897-44c2-8d5c-6e28dd99f65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:23.566252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e3b4b-1b8f-4447-b893-7de98ce12a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:23.566850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e3b4b-1b8f-4447-b893-7de98ce12a8a admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.567580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e3b4b-1b8f-4447-b893-7de98ce12a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.567875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e3b4b-1b8f-4447-b893-7de98ce12a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.569108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e3b4b-1b8f-4447-b893-7de98ce12a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:23.569425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e3b4b-1b8f-4447-b893-7de98ce12a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:23.572090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e3b4b-1b8f-4447-b893-7de98ce12a8a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:23.591867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 856/1732] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:01:23.596179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e3b4b-1b8f-4447-b893-7de98ce12a8a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:23.607391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e3b4b-1b8f-4447-b893-7de98ce12a8a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:23.615441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc721-7384-45ea-aef3-173d40fbf43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:23.616123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc721-7384-45ea-aef3-173d40fbf43c admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.616448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc721-7384-45ea-aef3-173d40fbf43c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.617271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc721-7384-45ea-aef3-173d40fbf43c admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.618249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfc721-7384-45ea-aef3-173d40fbf43c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:23.620112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfc721-7384-45ea-aef3-173d40fbf43c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:23.621924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:23.652168 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 877/1733] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:23.660905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09927d0a-9897-44c2-8d5c-6e28dd99f65f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:23.663692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-09927d0a-9897-44c2-8d5c-6e28dd99f65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:23.736617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 857/1734] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:23 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:23.737553 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc721-7384-45ea-aef3-173d40fbf43c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:23.750827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09927d0a-9897-44c2-8d5c-6e28dd99f65f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:23.803149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bb7a6-9d8d-4435-b403-c8e7b63c3f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:23.805371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bb7a6-9d8d-4435-b403-c8e7b63c3f0f service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.805742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bb7a6-9d8d-4435-b403-c8e7b63c3f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.808897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bb7a6-9d8d-4435-b403-c8e7b63c3f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.815776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc721-7384-45ea-aef3-173d40fbf43c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:23.822031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39926bee-47b5-4f83-8a5b-8162797439e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:23.823232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39926bee-47b5-4f83-8a5b-8162797439e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.823853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39926bee-47b5-4f83-8a5b-8162797439e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.824277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39926bee-47b5-4f83-8a5b-8162797439e0 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.826376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39926bee-47b5-4f83-8a5b-8162797439e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:23.829315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39926bee-47b5-4f83-8a5b-8162797439e0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:23.886227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bb7a6-9d8d-4435-b403-c8e7b63c3f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:23.888014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bb7a6-9d8d-4435-b403-c8e7b63c3f0f service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:23.900824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 878/1735] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:23.908442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 858/1736] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:23 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:23.911242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bb7a6-9d8d-4435-b403-c8e7b63c3f0f service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:23.925935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bb7a6-9d8d-4435-b403-c8e7b63c3f0f service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:23.931789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999b06-302e-4840-b707-2bcd6bbd855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:23.932502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999b06-302e-4840-b707-2bcd6bbd855d admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.932816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999b06-302e-4840-b707-2bcd6bbd855d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.937349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999b06-302e-4840-b707-2bcd6bbd855d admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.937349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999b06-302e-4840-b707-2bcd6bbd855d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:23.937349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999b06-302e-4840-b707-2bcd6bbd855d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:23.948683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 879/1737] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:23.950446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39926bee-47b5-4f83-8a5b-8162797439e0 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:23.961688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39926bee-47b5-4f83-8a5b-8162797439e0 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:23.969087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1610f-cbe2-4329-abb7-2d702582f7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:23.969782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1610f-cbe2-4329-abb7-2d702582f7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:23.970072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1610f-cbe2-4329-abb7-2d702582f7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:23.970350 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1610f-cbe2-4329-abb7-2d702582f7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:23.971198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1610f-cbe2-4329-abb7-2d702582f7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:23.972746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1610f-cbe2-4329-abb7-2d702582f7e6 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:23.984740 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 859/1738] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:23.989350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999b06-302e-4840-b707-2bcd6bbd855d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:24.006803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999b06-302e-4840-b707-2bcd6bbd855d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:24.013362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009c4e1-df97-4329-9337-ee42cb9c0cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:24.014525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009c4e1-df97-4329-9337-ee42cb9c0cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:24.014880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009c4e1-df97-4329-9337-ee42cb9c0cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:24.015224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009c4e1-df97-4329-9337-ee42cb9c0cdc admin admin] PATH_INFO: `/v3/projects/d433f409f9b14f928953c72559e139cd/users/497a01e93baa420d845628174b8726ef/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:24.020054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1610f-cbe2-4329-abb7-2d702582f7e6 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:24.029930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1610f-cbe2-4329-abb7-2d702582f7e6 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:24.032268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009c4e1-df97-4329-9337-ee42cb9c0cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d433f409f9b14f928953c72559e139cd, user_id=497a01e93baa420d845628174b8726ef, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:24.033614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4ed3b-1659-4e72-92b1-b6f4b957cd5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:24.034279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4ed3b-1659-4e72-92b1-b6f4b957cd5c service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:24.034570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4ed3b-1659-4e72-92b1-b6f4b957cd5c service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:24.034846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4ed3b-1659-4e72-92b1-b6f4b957cd5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:24.035169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009c4e1-df97-4329-9337-ee42cb9c0cdc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:24.036607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e6e4ed3b-1659-4e72-92b1-b6f4b957cd5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:24.109097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4ed3b-1659-4e72-92b1-b6f4b957cd5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:24.110727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4ed3b-1659-4e72-92b1-b6f4b957cd5c service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:24.122980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 880/1739] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:23 2022] PUT /identity/v3/projects/d433f409f9b14f928953c72559e139cd/users/497a01e93baa420d845628174b8726ef/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:24.129742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290f343-18ad-42f8-b3b2-6e59843b80f1 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:24.130544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290f343-18ad-42f8-b3b2-6e59843b80f1 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:24.130948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290f343-18ad-42f8-b3b2-6e59843b80f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:24.134441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 860/1740] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:01:24.173964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-9290f343-18ad-42f8-b3b2-6e59843b80f1 None None] MFA Rules not processed for user `497a01e93baa420d845628174b8726ef`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:24.176715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9290f343-18ad-42f8-b3b2-6e59843b80f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:24.327094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 881/1741] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:24.331457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4ed3b-1659-4e72-92b1-b6f4b957cd5c service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:24.342789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4ed3b-1659-4e72-92b1-b6f4b957cd5c service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:24.402340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aa688-4267-4fbf-819c-063321b30043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:24.403010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aa688-4267-4fbf-819c-063321b30043 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:24.403287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aa688-4267-4fbf-819c-063321b30043 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:24.403566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aa688-4267-4fbf-819c-063321b30043 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:24.404306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aa688-4267-4fbf-819c-063321b30043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:24.406245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aa688-4267-4fbf-819c-063321b30043 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:24.523326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 861/1742] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:24 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:24.530591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290f343-18ad-42f8-b3b2-6e59843b80f1 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:24.548158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290f343-18ad-42f8-b3b2-6e59843b80f1 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:24.613177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ad206-205d-41c6-b4ae-c845b4128d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:24.614264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ad206-205d-41c6-b4ae-c845b4128d96 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:24.622727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ad206-205d-41c6-b4ae-c845b4128d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:24.623029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ad206-205d-41c6-b4ae-c845b4128d96 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:24.623784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ad206-205d-41c6-b4ae-c845b4128d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:24.625859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ad206-205d-41c6-b4ae-c845b4128d96 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:24.721904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:24.751469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 882/1743] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:24 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:24.753263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aa688-4267-4fbf-819c-063321b30043 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:24.764810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aa688-4267-4fbf-819c-063321b30043 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:24.770591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db82cd-bdae-4280-9f3b-79a7a856ad59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:24.771482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db82cd-bdae-4280-9f3b-79a7a856ad59 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:24.772510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db82cd-bdae-4280-9f3b-79a7a856ad59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:24.773522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db82cd-bdae-4280-9f3b-79a7a856ad59 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:24.774559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db82cd-bdae-4280-9f3b-79a7a856ad59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:24.776853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db82cd-bdae-4280-9f3b-79a7a856ad59 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:24.791820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 862/1744] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:24.793380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ad206-205d-41c6-b4ae-c845b4128d96 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:24.803942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ad206-205d-41c6-b4ae-c845b4128d96 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:24.809444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d8e04-fdd4-40a3-8aa9-c42169939622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:24.810461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d8e04-fdd4-40a3-8aa9-c42169939622 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:24.810838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d8e04-fdd4-40a3-8aa9-c42169939622 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:24.811125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d8e04-fdd4-40a3-8aa9-c42169939622 admin admin] PATH_INFO: `/v3/projects/3dcd9a9f615b41cfb5decdd8aad9b4b3/users/d798a8cdd5c94339b327de6c161eb193/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:24.829267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d8e04-fdd4-40a3-8aa9-c42169939622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dcd9a9f615b41cfb5decdd8aad9b4b3, user_id=d798a8cdd5c94339b327de6c161eb193, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:24.830539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d8e04-fdd4-40a3-8aa9-c42169939622 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:24.957620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 883/1745] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:24 2022] PUT /identity/v3/projects/3dcd9a9f615b41cfb5decdd8aad9b4b3/users/d798a8cdd5c94339b327de6c161eb193/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:24.962190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db82cd-bdae-4280-9f3b-79a7a856ad59 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:24.972331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db82cd-bdae-4280-9f3b-79a7a856ad59 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:25.057102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe84fa9-e321-4908-ac3b-5546cafb8a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:25.058447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe84fa9-e321-4908-ac3b-5546cafb8a1a admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:25.058731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe84fa9-e321-4908-ac3b-5546cafb8a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:25.059411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe84fa9-e321-4908-ac3b-5546cafb8a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:25.060395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe84fa9-e321-4908-ac3b-5546cafb8a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:25.064171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe84fa9-e321-4908-ac3b-5546cafb8a1a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:25.074845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 863/1746] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:25.080983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d8e04-fdd4-40a3-8aa9-c42169939622 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:25.098394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d8e04-fdd4-40a3-8aa9-c42169939622 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:25.104903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a00cbd-2d9d-4af1-b469-7a3ef9095842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:25.105935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a00cbd-2d9d-4af1-b469-7a3ef9095842 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:25.106254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a00cbd-2d9d-4af1-b469-7a3ef9095842 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:25.106537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a00cbd-2d9d-4af1-b469-7a3ef9095842 admin admin] PATH_INFO: `/v3/projects/3dcd9a9f615b41cfb5decdd8aad9b4b3/users/d798a8cdd5c94339b327de6c161eb193/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:25.108960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a00cbd-2d9d-4af1-b469-7a3ef9095842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dcd9a9f615b41cfb5decdd8aad9b4b3, user_id=d798a8cdd5c94339b327de6c161eb193, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:25.112119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a00cbd-2d9d-4af1-b469-7a3ef9095842 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:25.156242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 884/1747] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:25 2022] PUT /identity/v3/projects/3dcd9a9f615b41cfb5decdd8aad9b4b3/users/d798a8cdd5c94339b327de6c161eb193/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:25.172624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e146c1-4329-4648-aaff-616127de656e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:25.172957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e146c1-4329-4648-aaff-616127de656e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:25.173242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e146c1-4329-4648-aaff-616127de656e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:25.205832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e3e146c1-4329-4648-aaff-616127de656e None None] MFA Rules not processed for user `d798a8cdd5c94339b327de6c161eb193`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:25.207076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e3e146c1-4329-4648-aaff-616127de656e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:25.254163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:25.287852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 864/1748] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:25.302176 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a00cbd-2d9d-4af1-b469-7a3ef9095842 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:25.306634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-42a00cbd-2d9d-4af1-b469-7a3ef9095842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:25.390872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a00cbd-2d9d-4af1-b469-7a3ef9095842 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:25.395402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74423904-dd31-4e3f-9c92-1ecf9603c094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:25.396287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74423904-dd31-4e3f-9c92-1ecf9603c094 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:25.396579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74423904-dd31-4e3f-9c92-1ecf9603c094 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:25.396898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74423904-dd31-4e3f-9c92-1ecf9603c094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:25.407884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74423904-dd31-4e3f-9c92-1ecf9603c094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:25.411736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74423904-dd31-4e3f-9c92-1ecf9603c094 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:25.424070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 885/1749] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:25.729715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e146c1-4329-4648-aaff-616127de656e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:25.742888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e146c1-4329-4648-aaff-616127de656e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:25.747022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810c790-2a27-4382-a04a-543f16e704dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:25.747888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810c790-2a27-4382-a04a-543f16e704dd service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:25.748378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810c790-2a27-4382-a04a-543f16e704dd service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:25.749117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810c790-2a27-4382-a04a-543f16e704dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:25.830037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810c790-2a27-4382-a04a-543f16e704dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:25.831752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810c790-2a27-4382-a04a-543f16e704dd service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:25.884204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 865/1750] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:26.615184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:26.787362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:27.423558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:27.533690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74423904-dd31-4e3f-9c92-1ecf9603c094 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:27.645092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74423904-dd31-4e3f-9c92-1ecf9603c094 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:27.654155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:27.655338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:27.655636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:27.655919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] PATH_INFO: `/v3/users/0b85604bb02744d283ab904e8dea2d29` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:27.657646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b85604bb02744d283ab904e8dea2d29)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:27.658780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: WARNING py.warnings [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 24 20:01:27.658780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: warnings.warn(msg) Jul 24 20:01:27.658780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: Jul 24 20:01:27.660298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:27.770329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459088d5-cff6-42cb-a119-3ae4ed4c320f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:27.774537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459088d5-cff6-42cb-a119-3ae4ed4c320f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:27.774537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459088d5-cff6-42cb-a119-3ae4ed4c320f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:27.864401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-459088d5-cff6-42cb-a119-3ae4ed4c320f None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:27.866616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-459088d5-cff6-42cb-a119-3ae4ed4c320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:27.899499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:27.994945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 866/1751] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:01:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:28.040595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b85604bb02744d283ab904e8dea2d29'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:28.064352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b85604bb02744d283ab904e8dea2d29'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:28.089182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b85604bb02744d283ab904e8dea2d29'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:28.093105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b85604bb02744d283ab904e8dea2d29'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:28.133944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:28.192925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b85604bb02744d283ab904e8dea2d29'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:28.214538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 886/1752] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:27 2022] DELETE /identity/v3/users/0b85604bb02744d283ab904e8dea2d29 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:28.221989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905fded-7337-4ecb-b265-709787f0ce0f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:28.222824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905fded-7337-4ecb-b265-709787f0ce0f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:28.223186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905fded-7337-4ecb-b265-709787f0ce0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:28.287564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e905fded-7337-4ecb-b265-709787f0ce0f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:28.293207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e905fded-7337-4ecb-b265-709787f0ce0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:28.475540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:28.477781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:28.533628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 867/1753] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:28.588321 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905fded-7337-4ecb-b265-709787f0ce0f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:28.590271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e905fded-7337-4ecb-b265-709787f0ce0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:28.725107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d2228-06e2-4024-bc57-e0983ea73067 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:28.727183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7f120-789a-4896-ba21-d579d6f512b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:28.727952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f120-789a-4896-ba21-d579d6f512b8 service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:28.728320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f120-789a-4896-ba21-d579d6f512b8 service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:28.728836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f120-789a-4896-ba21-d579d6f512b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:28.731248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-26f7f120-789a-4896-ba21-d579d6f512b8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:28.745980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905fded-7337-4ecb-b265-709787f0ce0f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:28.758846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5afb9-568f-414f-b32c-54ca5b304076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:28.759864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5afb9-568f-414f-b32c-54ca5b304076 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:28.760338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5afb9-568f-414f-b32c-54ca5b304076 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:28.760821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5afb9-568f-414f-b32c-54ca5b304076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:28.783064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5afb9-568f-414f-b32c-54ca5b304076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:28.790763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5afb9-568f-414f-b32c-54ca5b304076 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:28.810178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7f120-789a-4896-ba21-d579d6f512b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:28.812102 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7f120-789a-4896-ba21-d579d6f512b8 service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:28.813149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 868/1754] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:01:28.823049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 887/1755] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:01:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:01:28.884520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:29.015369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:29.491043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5afb9-568f-414f-b32c-54ca5b304076 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:29.492992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9bf5afb9-568f-414f-b32c-54ca5b304076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:29.517236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5afb9-568f-414f-b32c-54ca5b304076 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:29.524416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:29.525377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:29.534467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:29.534467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] PATH_INFO: `/v3/users/249bb2916e1a4257b8c2aeb8eee99191` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:29.534467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=249bb2916e1a4257b8c2aeb8eee99191)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:29.534467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:29.950289 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '249bb2916e1a4257b8c2aeb8eee99191'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:29.969160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '249bb2916e1a4257b8c2aeb8eee99191'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:29.973488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '249bb2916e1a4257b8c2aeb8eee99191'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:30.004511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeb461-e7c1-49a3-9ffe-36ec76ce8d2b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:30.005098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeb461-e7c1-49a3-9ffe-36ec76ce8d2b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:30.005473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeb461-e7c1-49a3-9ffe-36ec76ce8d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:30.048987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '249bb2916e1a4257b8c2aeb8eee99191'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:30.056120 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-feaeb461-e7c1-49a3-9ffe-36ec76ce8d2b None None] MFA Rules not processed for user `b9763f9899214df3929f3816f009e857`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:30.060002 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-feaeb461-e7c1-49a3-9ffe-36ec76ce8d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:30.088582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 888/1756] 10.222.0.42 () {58 vars in 1015 bytes} [Sun Jul 24 20:01:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:30.105038 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '249bb2916e1a4257b8c2aeb8eee99191'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:30.166116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 869/1757] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:29 2022] DELETE /identity/v3/users/249bb2916e1a4257b8c2aeb8eee99191 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:30.170813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570dc6f-a004-46ef-b409-e84bac8ef769 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:30.171879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570dc6f-a004-46ef-b409-e84bac8ef769 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:30.172393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570dc6f-a004-46ef-b409-e84bac8ef769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:30.220279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-7570dc6f-a004-46ef-b409-e84bac8ef769 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:30.232457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7570dc6f-a004-46ef-b409-e84bac8ef769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:30.363194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 889/1758] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:30.387679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:30.389235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:30.464920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570dc6f-a004-46ef-b409-e84bac8ef769 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:30.466724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7570dc6f-a004-46ef-b409-e84bac8ef769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:30.487743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570dc6f-a004-46ef-b409-e84bac8ef769 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:30.493500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:30.501271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:30.501271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:30.501271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] PATH_INFO: `/v3/projects/e04b9dac428e438bab45a825c81d441d` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:30.501271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e04b9dac428e438bab45a825c81d441d)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:30.501271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:30.602364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9aa9-0303-4940-a3c1-ec32707b8546 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:30.602364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6514935-ad4a-4210-b84b-4145c2450dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:30.602364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6514935-ad4a-4210-b84b-4145c2450dde service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:30.602364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6514935-ad4a-4210-b84b-4145c2450dde service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:30.602814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6514935-ad4a-4210-b84b-4145c2450dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:30.622864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6514935-ad4a-4210-b84b-4145c2450dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:30.624399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6514935-ad4a-4210-b84b-4145c2450dde service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:30.640300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 870/1759] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:30.718044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e04b9dac428e438bab45a825c81d441d', 'request_id': 'req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:30.722663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e04b9dac428e438bab45a825c81d441d', 'request_id': 'req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:30.731416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e04b9dac428e438bab45a825c81d441d', 'request_id': 'req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:30.872547 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] The token cache is being invalidate because project e04b9dac428e438bab45a825c81d441d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:30.880299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:30.880299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:30.882807 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 890/1760] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:30 2022] DELETE /identity/v3/projects/e04b9dac428e438bab45a825c81d441d => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:30.887003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6514935-ad4a-4210-b84b-4145c2450dde service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:30.895013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:30.900365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6514935-ad4a-4210-b84b-4145c2450dde service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:30.964576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:30.965518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:30.966835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:30.966835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] PATH_INFO: `/v3/users/3e8bb4b1562b4f9f8013939e050033fd` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:30.967292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e8bb4b1562b4f9f8013939e050033fd)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:30.969989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:31.210562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e8bb4b1562b4f9f8013939e050033fd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.210562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e8bb4b1562b4f9f8013939e050033fd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.210562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e8bb4b1562b4f9f8013939e050033fd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.347609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:31.362682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:31.396329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fc6f6-b6d7-43c1-bc63-df860b4658e6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:31.402234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:31.403827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:31.404537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:31.405453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] PATH_INFO: `/v3/projects/b6b7c24ef86e420fb3e454ef5a185551` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:31.409597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b7c24ef86e420fb3e454ef5a185551)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:31.412515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:31.448078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e8bb4b1562b4f9f8013939e050033fd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.480454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a97c831-9684-40da-a213-d71e9ac94ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e8bb4b1562b4f9f8013939e050033fd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.489086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 871/1761] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:30 2022] DELETE /identity/v3/users/3e8bb4b1562b4f9f8013939e050033fd => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:31.642960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b7c24ef86e420fb3e454ef5a185551', 'request_id': 'req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.649485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b7c24ef86e420fb3e454ef5a185551', 'request_id': 'req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.656200 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b7c24ef86e420fb3e454ef5a185551', 'request_id': 'req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.730060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] The token cache is being invalidate because project b6b7c24ef86e420fb3e454ef5a185551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:31.735861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.735861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef6fb1e7-fa5a-4f3f-a17e-47e2f8557412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:31.735861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 891/1762] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:31 2022] DELETE /identity/v3/projects/b6b7c24ef86e420fb3e454ef5a185551 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:31.738663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e881cc-0eea-4549-b2da-9c00b48b8389 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:31.738944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e881cc-0eea-4549-b2da-9c00b48b8389 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:31.739382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e881cc-0eea-4549-b2da-9c00b48b8389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:31.743332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:31.771463 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-18e881cc-0eea-4549-b2da-9c00b48b8389 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:31.775397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-18e881cc-0eea-4549-b2da-9c00b48b8389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:31.902428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 872/1763] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:31.909922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b46655-2895-4430-a011-24c588d6cc4c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:31.910494 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b46655-2895-4430-a011-24c588d6cc4c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:31.910879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b46655-2895-4430-a011-24c588d6cc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:31.938471 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-18b46655-2895-4430-a011-24c588d6cc4c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:31.940080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-18b46655-2895-4430-a011-24c588d6cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:31.962943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 892/1764] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:31.969397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e881cc-0eea-4549-b2da-9c00b48b8389 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:31.986950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e881cc-0eea-4549-b2da-9c00b48b8389 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:31.992680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de0777-bade-424a-9059-4cad55d92483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:31.993807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de0777-bade-424a-9059-4cad55d92483 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:31.994104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de0777-bade-424a-9059-4cad55d92483 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:31.994413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de0777-bade-424a-9059-4cad55d92483 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:31.995350 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de0777-bade-424a-9059-4cad55d92483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:31.995670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de0777-bade-424a-9059-4cad55d92483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:31.997349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de0777-bade-424a-9059-4cad55d92483 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:32.023866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 873/1765] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:01:32.028444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b46655-2895-4430-a011-24c588d6cc4c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:32.040066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b46655-2895-4430-a011-24c588d6cc4c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:32.047241 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdf6a8-e279-4219-9be4-cbd895597167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:32.050317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdf6a8-e279-4219-9be4-cbd895597167 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:32.050317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdf6a8-e279-4219-9be4-cbd895597167 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:32.050317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdf6a8-e279-4219-9be4-cbd895597167 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:32.050317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdf6a8-e279-4219-9be4-cbd895597167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:32.062770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdf6a8-e279-4219-9be4-cbd895597167 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:32.154204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 893/1766] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:32 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:32.158648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de0777-bade-424a-9059-4cad55d92483 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:32.177743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de0777-bade-424a-9059-4cad55d92483 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:32.287292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aacdaa-1fb0-4dd2-b7a1-c68df4f5cc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:32.288684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aacdaa-1fb0-4dd2-b7a1-c68df4f5cc8b admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:32.288999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aacdaa-1fb0-4dd2-b7a1-c68df4f5cc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:32.289288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aacdaa-1fb0-4dd2-b7a1-c68df4f5cc8b admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:32.290173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aacdaa-1fb0-4dd2-b7a1-c68df4f5cc8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:32.297895 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdf6a8-e279-4219-9be4-cbd895597167 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:32.305721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-90fdf6a8-e279-4219-9be4-cbd895597167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:32.306154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aacdaa-1fb0-4dd2-b7a1-c68df4f5cc8b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:32.320267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdf6a8-e279-4219-9be4-cbd895597167 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:32.326254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:32.328196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:32.329008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:32.329752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] PATH_INFO: `/v3/projects/71b96e36b4f8424a982bb50ba0d0ddb4` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:32.333543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b96e36b4f8424a982bb50ba0d0ddb4)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:32.336675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:32.805210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 874/1767] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:32 2022] POST /identity/v3/users => generated 287 bytes in 647 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:32.810571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aacdaa-1fb0-4dd2-b7a1-c68df4f5cc8b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:32.817970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b96e36b4f8424a982bb50ba0d0ddb4', 'request_id': 'req-572c0074-d158-4199-ad59-3b8da8a90478'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:32.825282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aacdaa-1fb0-4dd2-b7a1-c68df4f5cc8b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:32.825509 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b96e36b4f8424a982bb50ba0d0ddb4', 'request_id': 'req-572c0074-d158-4199-ad59-3b8da8a90478'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:32.833131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b96e36b4f8424a982bb50ba0d0ddb4', 'request_id': 'req-572c0074-d158-4199-ad59-3b8da8a90478'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:32.917079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c489315-1186-463d-b463-41895155114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:32.918079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c489315-1186-463d-b463-41895155114c admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:32.918508 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c489315-1186-463d-b463-41895155114c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:32.918872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c489315-1186-463d-b463-41895155114c admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:32.919778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c489315-1186-463d-b463-41895155114c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:32.922121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c489315-1186-463d-b463-41895155114c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:32.935442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 875/1768] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:32.940747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c489315-1186-463d-b463-41895155114c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:33.056345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c489315-1186-463d-b463-41895155114c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:33.056345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4c5bf-0ec7-4379-ae14-600b801ddb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:33.056345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4c5bf-0ec7-4379-ae14-600b801ddb21 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:33.056345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4c5bf-0ec7-4379-ae14-600b801ddb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:33.056876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4c5bf-0ec7-4379-ae14-600b801ddb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:33.056876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4c5bf-0ec7-4379-ae14-600b801ddb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:33.056876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4c5bf-0ec7-4379-ae14-600b801ddb21 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:33.056876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 876/1769] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:33.064703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4c5bf-0ec7-4379-ae14-600b801ddb21 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:33.075721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4c5bf-0ec7-4379-ae14-600b801ddb21 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:33.083314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9983d8-60b2-45d3-bd97-368394a76107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:33.084248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9983d8-60b2-45d3-bd97-368394a76107 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:33.084669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9983d8-60b2-45d3-bd97-368394a76107 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:33.085065 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9983d8-60b2-45d3-bd97-368394a76107 admin admin] PATH_INFO: `/v3/projects/ba0a6e1b9a2b43a097a0b25f913ad2d9/users/b723d57a7dc44b4d8f8af2a9f47c614f/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:33.111384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9983d8-60b2-45d3-bd97-368394a76107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0a6e1b9a2b43a097a0b25f913ad2d9, user_id=b723d57a7dc44b4d8f8af2a9f47c614f, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:33.114893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9983d8-60b2-45d3-bd97-368394a76107 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:33.271063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] The token cache is being invalidate because project 71b96e36b4f8424a982bb50ba0d0ddb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:33.271858 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:33.273709 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-572c0074-d158-4199-ad59-3b8da8a90478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:33.276543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 894/1770] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:32 2022] DELETE /identity/v3/projects/71b96e36b4f8424a982bb50ba0d0ddb4 => generated 0 bytes in 981 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:33.281680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111379d6-81af-449f-9088-0b5d428db879 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:33.281982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111379d6-81af-449f-9088-0b5d428db879 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:33.283800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 877/1771] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:33 2022] PUT /identity/v3/projects/ba0a6e1b9a2b43a097a0b25f913ad2d9/users/b723d57a7dc44b4d8f8af2a9f47c614f/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:33.293061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111379d6-81af-449f-9088-0b5d428db879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:33.296943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64d0e-ff55-44dc-9cd8-6f2c0bc9104b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:33.297841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64d0e-ff55-44dc-9cd8-6f2c0bc9104b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:33.298277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64d0e-ff55-44dc-9cd8-6f2c0bc9104b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:33.348219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6dd64d0e-ff55-44dc-9cd8-6f2c0bc9104b None None] MFA Rules not processed for user `b723d57a7dc44b4d8f8af2a9f47c614f`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:33.349810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6dd64d0e-ff55-44dc-9cd8-6f2c0bc9104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:33.352145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-111379d6-81af-449f-9088-0b5d428db879 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:33.353382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-111379d6-81af-449f-9088-0b5d428db879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:33.447827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 878/1772] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:33.453568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64d0e-ff55-44dc-9cd8-6f2c0bc9104b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:33.455423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6dd64d0e-ff55-44dc-9cd8-6f2c0bc9104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:33.461427 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 895/1773] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:33.474989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e961097-6e65-464b-af9d-13a8b97db5d1 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:33.478434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e961097-6e65-464b-af9d-13a8b97db5d1 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:33.478434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e961097-6e65-464b-af9d-13a8b97db5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:33.509431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64d0e-ff55-44dc-9cd8-6f2c0bc9104b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:33.525218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631e2fc-9bb3-427a-9cec-88a185daa0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:33.525916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631e2fc-9bb3-427a-9cec-88a185daa0fc admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:33.526188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631e2fc-9bb3-427a-9cec-88a185daa0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:33.526470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631e2fc-9bb3-427a-9cec-88a185daa0fc admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:33.527545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631e2fc-9bb3-427a-9cec-88a185daa0fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:33.529458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631e2fc-9bb3-427a-9cec-88a185daa0fc admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:33.541915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-8e961097-6e65-464b-af9d-13a8b97db5d1 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:33.544287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8e961097-6e65-464b-af9d-13a8b97db5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:33.576991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 896/1774] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:33.581723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e961097-6e65-464b-af9d-13a8b97db5d1 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:33.594101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e961097-6e65-464b-af9d-13a8b97db5d1 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:33.599166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848a1db-1d4e-4981-af42-d2176735e23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:33.599841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848a1db-1d4e-4981-af42-d2176735e23c admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:33.600129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848a1db-1d4e-4981-af42-d2176735e23c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:33.600431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848a1db-1d4e-4981-af42-d2176735e23c admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:33.601335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848a1db-1d4e-4981-af42-d2176735e23c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:33.601619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848a1db-1d4e-4981-af42-d2176735e23c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:33.603362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848a1db-1d4e-4981-af42-d2176735e23c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:33.617728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 897/1775] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:01:33.622312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848a1db-1d4e-4981-af42-d2176735e23c admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:33.639124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848a1db-1d4e-4981-af42-d2176735e23c admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:33.645426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917d838-8216-4caf-b66a-ec7733af31db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:33.649812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d838-8216-4caf-b66a-ec7733af31db admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:33.649959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 879/1776] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:33 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:33.650067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d838-8216-4caf-b66a-ec7733af31db admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:33.650067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d838-8216-4caf-b66a-ec7733af31db admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:33.650067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917d838-8216-4caf-b66a-ec7733af31db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:33.650888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917d838-8216-4caf-b66a-ec7733af31db admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:33.651000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631e2fc-9bb3-427a-9cec-88a185daa0fc admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:33.666764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:33.739515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631e2fc-9bb3-427a-9cec-88a185daa0fc admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:33.745217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f985009-3d3e-4f1b-ab5b-1445d2e4d71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:33.746033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f985009-3d3e-4f1b-ab5b-1445d2e4d71b admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:33.746402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f985009-3d3e-4f1b-ab5b-1445d2e4d71b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:33.746755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f985009-3d3e-4f1b-ab5b-1445d2e4d71b admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:33.747598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f985009-3d3e-4f1b-ab5b-1445d2e4d71b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:33.750666 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f985009-3d3e-4f1b-ab5b-1445d2e4d71b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:34.003630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 898/1777] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:33 2022] POST /identity/v3/projects => generated 375 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:34.005262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917d838-8216-4caf-b66a-ec7733af31db admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:34.060131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 880/1778] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:33 2022] POST /identity/v3/users => generated 293 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:34.062671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f985009-3d3e-4f1b-ab5b-1445d2e4d71b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:34.096858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f985009-3d3e-4f1b-ab5b-1445d2e4d71b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:34.107367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992fee1-08fc-4291-a06f-4000ff307665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:34.108174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992fee1-08fc-4291-a06f-4000ff307665 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:34.108348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917d838-8216-4caf-b66a-ec7733af31db admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:34.108488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992fee1-08fc-4291-a06f-4000ff307665 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:34.112051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992fee1-08fc-4291-a06f-4000ff307665 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:34.113660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992fee1-08fc-4291-a06f-4000ff307665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:34.115804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992fee1-08fc-4291-a06f-4000ff307665 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:34.119322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed91df7-645e-4fb9-9edf-c8aa666a7ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:34.120522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed91df7-645e-4fb9-9edf-c8aa666a7ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:34.127706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed91df7-645e-4fb9-9edf-c8aa666a7ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:34.127706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed91df7-645e-4fb9-9edf-c8aa666a7ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:34.127706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed91df7-645e-4fb9-9edf-c8aa666a7ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:34.130957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed91df7-645e-4fb9-9edf-c8aa666a7ba8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:34.157651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 881/1779] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:34.163203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992fee1-08fc-4291-a06f-4000ff307665 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:34.182873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992fee1-08fc-4291-a06f-4000ff307665 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:34.196797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7bc3a-d0dd-4c53-8595-20a4cc5de02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:34.201981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7bc3a-d0dd-4c53-8595-20a4cc5de02f admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:34.201981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7bc3a-d0dd-4c53-8595-20a4cc5de02f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:34.201981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7bc3a-d0dd-4c53-8595-20a4cc5de02f admin admin] PATH_INFO: `/v3/projects/c74d98d06f6c4b1b867d075217f0a708/users/ef93fe8aebdf45e99b31ecd43c8c528f/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:34.237779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7bc3a-d0dd-4c53-8595-20a4cc5de02f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74d98d06f6c4b1b867d075217f0a708, user_id=ef93fe8aebdf45e99b31ecd43c8c528f, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:34.240571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7bc3a-d0dd-4c53-8595-20a4cc5de02f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:34.606942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 899/1780] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:34 2022] POST /identity/v3/users => generated 288 bytes in 602 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:34.610776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed91df7-645e-4fb9-9edf-c8aa666a7ba8 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:34.634924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 882/1781] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:34 2022] PUT /identity/v3/projects/c74d98d06f6c4b1b867d075217f0a708/users/ef93fe8aebdf45e99b31ecd43c8c528f/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:34.639304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7bc3a-d0dd-4c53-8595-20a4cc5de02f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:34.653297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed91df7-645e-4fb9-9edf-c8aa666a7ba8 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:34.742319 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1676d-5f6a-4ed9-89ed-c4d089a0423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:34.743145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1676d-5f6a-4ed9-89ed-c4d089a0423f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:34.743774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1676d-5f6a-4ed9-89ed-c4d089a0423f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:34.744064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1676d-5f6a-4ed9-89ed-c4d089a0423f admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:34.744918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1676d-5f6a-4ed9-89ed-c4d089a0423f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:34.746559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1676d-5f6a-4ed9-89ed-c4d089a0423f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:34.759468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 900/1782] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:34.764007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1676d-5f6a-4ed9-89ed-c4d089a0423f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:34.773785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:34.778751 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7bc3a-d0dd-4c53-8595-20a4cc5de02f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:34.783798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2e5aa-6468-4ddb-b9aa-d3117e2eb8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:34.784494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2e5aa-6468-4ddb-b9aa-d3117e2eb8cf admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:34.784864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2e5aa-6468-4ddb-b9aa-d3117e2eb8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:34.785151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2e5aa-6468-4ddb-b9aa-d3117e2eb8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:34.785897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2e5aa-6468-4ddb-b9aa-d3117e2eb8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:34.787524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2e5aa-6468-4ddb-b9aa-d3117e2eb8cf admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:34.796512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1676d-5f6a-4ed9-89ed-c4d089a0423f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:34.806148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 883/1783] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:34.810821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2e5aa-6468-4ddb-b9aa-d3117e2eb8cf admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:34.812974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1b32d-b887-428e-9089-e7b93ea7e18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:34.813742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1b32d-b887-428e-9089-e7b93ea7e18a admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:34.814083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1b32d-b887-428e-9089-e7b93ea7e18a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:34.814398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1b32d-b887-428e-9089-e7b93ea7e18a admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:34.815248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1b32d-b887-428e-9089-e7b93ea7e18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:34.817144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1b32d-b887-428e-9089-e7b93ea7e18a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:34.832386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2e5aa-6468-4ddb-b9aa-d3117e2eb8cf admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:34.832687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 901/1784] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:34.833679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1b32d-b887-428e-9089-e7b93ea7e18a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:34.838723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14094444-74e7-4f08-add0-73dd7febf21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:34.839896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14094444-74e7-4f08-add0-73dd7febf21e admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:34.840464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14094444-74e7-4f08-add0-73dd7febf21e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:34.840949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14094444-74e7-4f08-add0-73dd7febf21e admin admin] PATH_INFO: `/v3/projects/c74d98d06f6c4b1b867d075217f0a708/users/ef93fe8aebdf45e99b31ecd43c8c528f/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:34.843308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14094444-74e7-4f08-add0-73dd7febf21e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74d98d06f6c4b1b867d075217f0a708, user_id=ef93fe8aebdf45e99b31ecd43c8c528f, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:34.845273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1b32d-b887-428e-9089-e7b93ea7e18a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:34.846533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14094444-74e7-4f08-add0-73dd7febf21e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:34.851770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9168b-c933-4a71-8563-c05471a1c077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:34.852972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9168b-c933-4a71-8563-c05471a1c077 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:34.853318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9168b-c933-4a71-8563-c05471a1c077 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:34.853657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9168b-c933-4a71-8563-c05471a1c077 admin admin] PATH_INFO: `/v3/projects/cb774a313fd84e148308ed9fcf1a2512/users/765b9b950b42453b809c20e8c4db7f2b/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:34.870988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 884/1785] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:34 2022] PUT /identity/v3/projects/c74d98d06f6c4b1b867d075217f0a708/users/ef93fe8aebdf45e99b31ecd43c8c528f/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:34.883546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc6500-5b8a-43b7-8144-50c506bd2952 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:34.884108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc6500-5b8a-43b7-8144-50c506bd2952 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:34.884593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc6500-5b8a-43b7-8144-50c506bd2952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:34.885399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9168b-c933-4a71-8563-c05471a1c077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb774a313fd84e148308ed9fcf1a2512, user_id=765b9b950b42453b809c20e8c4db7f2b, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:34.888080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9168b-c933-4a71-8563-c05471a1c077 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:34.946927 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-58bc6500-5b8a-43b7-8144-50c506bd2952 None None] MFA Rules not processed for user `ef93fe8aebdf45e99b31ecd43c8c528f`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:34.946927 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-58bc6500-5b8a-43b7-8144-50c506bd2952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:35.013412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 885/1786] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:35.025309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc6500-5b8a-43b7-8144-50c506bd2952 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:35.026791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-58bc6500-5b8a-43b7-8144-50c506bd2952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:35.084470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc6500-5b8a-43b7-8144-50c506bd2952 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:35.088272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d2ee-e299-47dd-bf59-0c62a59b8124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:35.088992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d2ee-e299-47dd-bf59-0c62a59b8124 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:35.089285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d2ee-e299-47dd-bf59-0c62a59b8124 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:35.089569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d2ee-e299-47dd-bf59-0c62a59b8124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:35.098971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145d2ee-e299-47dd-bf59-0c62a59b8124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:35.101048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145d2ee-e299-47dd-bf59-0c62a59b8124 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:35.110771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 886/1787] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:01:35.111590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d2ee-e299-47dd-bf59-0c62a59b8124 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:35.112960 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a145d2ee-e299-47dd-bf59-0c62a59b8124 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:35.179895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d2ee-e299-47dd-bf59-0c62a59b8124 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:35.181534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 902/1788] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:34 2022] PUT /identity/v3/projects/cb774a313fd84e148308ed9fcf1a2512/users/765b9b950b42453b809c20e8c4db7f2b/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:35.192318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f57b31-8e35-4283-aebd-ea702b98d6a1 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:35.192507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f57b31-8e35-4283-aebd-ea702b98d6a1 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:35.192870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f57b31-8e35-4283-aebd-ea702b98d6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:35.222695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a3f57b31-8e35-4283-aebd-ea702b98d6a1 None None] MFA Rules not processed for user `765b9b950b42453b809c20e8c4db7f2b`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:35.224205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a3f57b31-8e35-4283-aebd-ea702b98d6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:35.241917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b860d-89de-493d-aafa-4bc4143093ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:35.241917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b860d-89de-493d-aafa-4bc4143093ed service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:35.241917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b860d-89de-493d-aafa-4bc4143093ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:35.241917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b860d-89de-493d-aafa-4bc4143093ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:35.243569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fd1b860d-89de-493d-aafa-4bc4143093ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:35.255830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b860d-89de-493d-aafa-4bc4143093ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:35.257336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b860d-89de-493d-aafa-4bc4143093ed service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:35.266026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 887/1789] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:35.299365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 903/1790] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:35.304555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b860d-89de-493d-aafa-4bc4143093ed service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:35.373693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b860d-89de-493d-aafa-4bc4143093ed service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:35.381206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41302514-41bf-4f71-8bc3-e16f009b9055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:35.381206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41302514-41bf-4f71-8bc3-e16f009b9055 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:35.381206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41302514-41bf-4f71-8bc3-e16f009b9055 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:35.381206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41302514-41bf-4f71-8bc3-e16f009b9055 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:35.381650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41302514-41bf-4f71-8bc3-e16f009b9055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:35.383042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41302514-41bf-4f71-8bc3-e16f009b9055 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:35.396215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f57b31-8e35-4283-aebd-ea702b98d6a1 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:35.466057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f57b31-8e35-4283-aebd-ea702b98d6a1 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:35.471728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d2c1f-c46e-424f-8a4e-1b99b44bde81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:35.473113 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d2c1f-c46e-424f-8a4e-1b99b44bde81 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:35.473731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d2c1f-c46e-424f-8a4e-1b99b44bde81 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:35.474248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d2c1f-c46e-424f-8a4e-1b99b44bde81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:35.484365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 888/1791] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:35 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:35.489055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41302514-41bf-4f71-8bc3-e16f009b9055 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:35.558599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d2c1f-c46e-424f-8a4e-1b99b44bde81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:35.560847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d2c1f-c46e-424f-8a4e-1b99b44bde81 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:35.571751 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41302514-41bf-4f71-8bc3-e16f009b9055 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:35.577898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 904/1792] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:01:35.587442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86f2df-25fe-4dd5-8c4c-65b1e5980598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:35.588060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86f2df-25fe-4dd5-8c4c-65b1e5980598 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:35.588871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86f2df-25fe-4dd5-8c4c-65b1e5980598 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:35.589314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86f2df-25fe-4dd5-8c4c-65b1e5980598 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:35.590303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86f2df-25fe-4dd5-8c4c-65b1e5980598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:35.594904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86f2df-25fe-4dd5-8c4c-65b1e5980598 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:35.710285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 889/1793] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:35 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:35.711669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d2c1f-c46e-424f-8a4e-1b99b44bde81 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:35.723570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d2c1f-c46e-424f-8a4e-1b99b44bde81 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:35.728455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0e913-3000-4054-814a-e056ca90a605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:35.729192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0e913-3000-4054-814a-e056ca90a605 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:35.729481 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0e913-3000-4054-814a-e056ca90a605 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:35.729759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0e913-3000-4054-814a-e056ca90a605 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:35.730493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0e913-3000-4054-814a-e056ca90a605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:35.733390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0e913-3000-4054-814a-e056ca90a605 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:35.747548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 905/1794] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:35.752048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86f2df-25fe-4dd5-8c4c-65b1e5980598 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:35.764795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86f2df-25fe-4dd5-8c4c-65b1e5980598 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:35.769859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61f5c-65aa-4793-b277-ac628a1ae1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:35.770901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61f5c-65aa-4793-b277-ac628a1ae1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:35.773528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61f5c-65aa-4793-b277-ac628a1ae1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:35.773528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61f5c-65aa-4793-b277-ac628a1ae1dd admin admin] PATH_INFO: `/v3/projects/ac050a1fca1848cbb0f3060de2c32888/users/4e09a0c721094a788b54badccc64ea25/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:35.790294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb61f5c-65aa-4793-b277-ac628a1ae1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac050a1fca1848cbb0f3060de2c32888, user_id=4e09a0c721094a788b54badccc64ea25, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:35.802291 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb61f5c-65aa-4793-b277-ac628a1ae1dd admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:35.873690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 890/1795] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:35 2022] PUT /identity/v3/projects/ac050a1fca1848cbb0f3060de2c32888/users/4e09a0c721094a788b54badccc64ea25/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:35.878098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0e913-3000-4054-814a-e056ca90a605 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:35.890798 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0e913-3000-4054-814a-e056ca90a605 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:35.964135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c042df6-e2d1-4e40-948a-a3d2ad3eebcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:35.964863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c042df6-e2d1-4e40-948a-a3d2ad3eebcd admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:35.965303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c042df6-e2d1-4e40-948a-a3d2ad3eebcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:35.965597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c042df6-e2d1-4e40-948a-a3d2ad3eebcd admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:35.966480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c042df6-e2d1-4e40-948a-a3d2ad3eebcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:35.968112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c042df6-e2d1-4e40-948a-a3d2ad3eebcd admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:35.979882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 906/1796] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:35.984047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61f5c-65aa-4793-b277-ac628a1ae1dd admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:35.997790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61f5c-65aa-4793-b277-ac628a1ae1dd admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:36.004673 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4416b2-bd84-4d50-a7f2-87fd467a119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:36.006260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4416b2-bd84-4d50-a7f2-87fd467a119a admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:36.006260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4416b2-bd84-4d50-a7f2-87fd467a119a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:36.006260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4416b2-bd84-4d50-a7f2-87fd467a119a admin admin] PATH_INFO: `/v3/projects/ac050a1fca1848cbb0f3060de2c32888/users/4e09a0c721094a788b54badccc64ea25/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:36.009180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4416b2-bd84-4d50-a7f2-87fd467a119a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac050a1fca1848cbb0f3060de2c32888, user_id=4e09a0c721094a788b54badccc64ea25, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:36.015042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4416b2-bd84-4d50-a7f2-87fd467a119a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:36.029590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 891/1797] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:35 2022] PUT /identity/v3/projects/ac050a1fca1848cbb0f3060de2c32888/users/4e09a0c721094a788b54badccc64ea25/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:36.040721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bb4fa-7a15-402d-b0bb-1f1e37ca53a4 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:36.041138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bb4fa-7a15-402d-b0bb-1f1e37ca53a4 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:36.041295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bb4fa-7a15-402d-b0bb-1f1e37ca53a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:36.109160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-586bb4fa-7a15-402d-b0bb-1f1e37ca53a4 None None] MFA Rules not processed for user `4e09a0c721094a788b54badccc64ea25`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:36.110443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-586bb4fa-7a15-402d-b0bb-1f1e37ca53a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:36.149586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:36.294577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 907/1798] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:36.307427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4416b2-bd84-4d50-a7f2-87fd467a119a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:36.310554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9b4416b2-bd84-4d50-a7f2-87fd467a119a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:36.404199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4416b2-bd84-4d50-a7f2-87fd467a119a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:36.417878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5f7c4-f482-4091-a11c-fc9e3c9d5b4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:36.417878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5f7c4-f482-4091-a11c-fc9e3c9d5b4f service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:36.417878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5f7c4-f482-4091-a11c-fc9e3c9d5b4f service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:36.417878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5f7c4-f482-4091-a11c-fc9e3c9d5b4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:36.513408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5f7c4-f482-4091-a11c-fc9e3c9d5b4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:36.514782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5f7c4-f482-4091-a11c-fc9e3c9d5b4f service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:36.548871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 892/1799] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:36.663852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f43d9-1834-43b4-a6f0-48185fec5263 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:36.664185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f43d9-1834-43b4-a6f0-48185fec5263 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:36.664419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f43d9-1834-43b4-a6f0-48185fec5263 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:36.667691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 908/1800] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:01:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:36.689707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ed525-931d-4a42-8910-9d9169ddf270 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:36.689707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ed525-931d-4a42-8910-9d9169ddf270 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:36.689881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ed525-931d-4a42-8910-9d9169ddf270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:36.757871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-4a9ed525-931d-4a42-8910-9d9169ddf270 None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:36.761128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4a9ed525-931d-4a42-8910-9d9169ddf270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:36.922127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 893/1801] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:01:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:36.947511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f43d9-1834-43b4-a6f0-48185fec5263 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:36.949578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9d8f43d9-1834-43b4-a6f0-48185fec5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:36.973873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f43d9-1834-43b4-a6f0-48185fec5263 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:36.981663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2060d5-7678-4e54-a8de-68d480263c51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:36.982376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2060d5-7678-4e54-a8de-68d480263c51 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:36.982854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2060d5-7678-4e54-a8de-68d480263c51 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:36.983353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2060d5-7678-4e54-a8de-68d480263c51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:37.018542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2060d5-7678-4e54-a8de-68d480263c51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:37.022706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2060d5-7678-4e54-a8de-68d480263c51 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:37.036116 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 909/1802] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:38.549261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:39.837054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e901bf-4f24-4654-b567-1b6aaf15dacd None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:39.837862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e901bf-4f24-4654-b567-1b6aaf15dacd None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:39.837862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e901bf-4f24-4654-b567-1b6aaf15dacd None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:39.838124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:39.838918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 894/1803] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:39.857569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869479fe-7163-4eb1-9622-8dca61e1e468 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:39.857875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869479fe-7163-4eb1-9622-8dca61e1e468 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:39.858158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869479fe-7163-4eb1-9622-8dca61e1e468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:39.889961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-869479fe-7163-4eb1-9622-8dca61e1e468 None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:39.893061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-869479fe-7163-4eb1-9622-8dca61e1e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:39.941262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 910/1804] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:01:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:40.000929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e901bf-4f24-4654-b567-1b6aaf15dacd None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:40.039793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e901bf-4f24-4654-b567-1b6aaf15dacd None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:40.056964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1221a7-ec3a-4106-8c04-8b8278a2e4d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:40.058395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1221a7-ec3a-4106-8c04-8b8278a2e4d2 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:40.058684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1221a7-ec3a-4106-8c04-8b8278a2e4d2 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:40.058969 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1221a7-ec3a-4106-8c04-8b8278a2e4d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:40.081048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1221a7-ec3a-4106-8c04-8b8278a2e4d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:40.083831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1221a7-ec3a-4106-8c04-8b8278a2e4d2 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:40.109247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 895/1805] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:41.465244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869479fe-7163-4eb1-9622-8dca61e1e468 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:41.467019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-869479fe-7163-4eb1-9622-8dca61e1e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:41.549084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869479fe-7163-4eb1-9622-8dca61e1e468 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:41.549084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:41.549084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:41.549084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:41.549634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] PATH_INFO: `/v3/users/702d182b40104b329076ed297d91edb6` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:41.550456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=702d182b40104b329076ed297d91edb6)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:41.552786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:41.766225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702d182b40104b329076ed297d91edb6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:41.778507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702d182b40104b329076ed297d91edb6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:41.785393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702d182b40104b329076ed297d91edb6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:41.786094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702d182b40104b329076ed297d91edb6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:41.864132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702d182b40104b329076ed297d91edb6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:41.874035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 911/1806] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:41 2022] DELETE /identity/v3/users/702d182b40104b329076ed297d91edb6 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:41.875766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3723a-ab71-485c-bfa0-df7a5d9e3010 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:41.876279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3723a-ab71-485c-bfa0-df7a5d9e3010 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:41.876374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3723a-ab71-485c-bfa0-df7a5d9e3010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:41.894310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-20e3723a-ab71-485c-bfa0-df7a5d9e3010 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:41.895518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-20e3723a-ab71-485c-bfa0-df7a5d9e3010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:41.953950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 896/1807] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:41.963536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:41.965596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:42.022632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5511c-5aa5-453e-95fd-b85b36759653 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:42.026607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452e80f-1e45-4a40-a1b7-8fb7e9a1d1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:42.027206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452e80f-1e45-4a40-a1b7-8fb7e9a1d1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:42.027495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452e80f-1e45-4a40-a1b7-8fb7e9a1d1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:42.027773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452e80f-1e45-4a40-a1b7-8fb7e9a1d1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:42.038479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452e80f-1e45-4a40-a1b7-8fb7e9a1d1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:42.039968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452e80f-1e45-4a40-a1b7-8fb7e9a1d1e6 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:42.060742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 912/1808] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:42.420577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:42.651086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3723a-ab71-485c-bfa0-df7a5d9e3010 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:42.652625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-20e3723a-ab71-485c-bfa0-df7a5d9e3010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:42.664967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3723a-ab71-485c-bfa0-df7a5d9e3010 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:42.669695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:42.670883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:42.671149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:42.671429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] PATH_INFO: `/v3/projects/2e432ae974854f22b41d5c922fa4bd86` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:42.672662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e432ae974854f22b41d5c922fa4bd86)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:42.674626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:42.704498 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:42.803030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e432ae974854f22b41d5c922fa4bd86', 'request_id': 'req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:42.809438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e432ae974854f22b41d5c922fa4bd86', 'request_id': 'req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:42.871633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e432ae974854f22b41d5c922fa4bd86', 'request_id': 'req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:42.883695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] The token cache is being invalidate because project 2e432ae974854f22b41d5c922fa4bd86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:42.884190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:42.884425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:42.886015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 897/1809] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:42 2022] DELETE /identity/v3/projects/2e432ae974854f22b41d5c922fa4bd86 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:42.890286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452e80f-1e45-4a40-a1b7-8fb7e9a1d1e6 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:42.900926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452e80f-1e45-4a40-a1b7-8fb7e9a1d1e6 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:42.961281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:42.962239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:42.962535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:42.962820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] PATH_INFO: `/v3/users/b4d7c8ea41384cb5809e15faa3edfb3e` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:42.964081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d7c8ea41384cb5809e15faa3edfb3e)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:42.967362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:43.003076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:43.199862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d7c8ea41384cb5809e15faa3edfb3e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:43.211646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d7c8ea41384cb5809e15faa3edfb3e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:43.218591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d7c8ea41384cb5809e15faa3edfb3e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:43.219329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d7c8ea41384cb5809e15faa3edfb3e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:43.292815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d7c8ea41384cb5809e15faa3edfb3e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:43.299691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 913/1810] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:42 2022] DELETE /identity/v3/users/b4d7c8ea41384cb5809e15faa3edfb3e => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:44.081676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:44.133192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:44.134642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:44.148612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:44.150218 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:44.150363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf359a8-0987-49d6-a24a-bd0f0ba136b0 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:44.155933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:44.211678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:44.212920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:44.213208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:44.213486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] PATH_INFO: `/v3/users/765b9b950b42453b809c20e8c4db7f2b` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:44.214731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=765b9b950b42453b809c20e8c4db7f2b)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:44.216636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:44.229532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce2e70-444c-4bac-884a-f93870cac6da admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:44.234594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:44.235448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:44.235736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:44.236016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] PATH_INFO: `/v3/projects/99606e335add4c5bb3573651418a298a` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:44.237247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] RBAC: Authorizing `identity:delete_project(project_id=99606e335add4c5bb3573651418a298a)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:44.239198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:44.432590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99606e335add4c5bb3573651418a298a', 'request_id': 'req-8b958e8a-3786-43a0-854a-96f93dce044b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.432904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765b9b950b42453b809c20e8c4db7f2b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.438042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99606e335add4c5bb3573651418a298a', 'request_id': 'req-8b958e8a-3786-43a0-854a-96f93dce044b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.440224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765b9b950b42453b809c20e8c4db7f2b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.440762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765b9b950b42453b809c20e8c4db7f2b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.445339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99606e335add4c5bb3573651418a298a', 'request_id': 'req-8b958e8a-3786-43a0-854a-96f93dce044b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.509827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765b9b950b42453b809c20e8c4db7f2b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.522250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1e64d92e-41ea-4bdc-a915-42214c4faf3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765b9b950b42453b809c20e8c4db7f2b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.528171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 898/1811] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:44 2022] DELETE /identity/v3/users/765b9b950b42453b809c20e8c4db7f2b => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:44.532537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17198b9a-ca68-4b61-b4aa-40ce9badb88c None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:44.533081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17198b9a-ca68-4b61-b4aa-40ce9badb88c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:44.533541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17198b9a-ca68-4b61-b4aa-40ce9badb88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:44.551936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-17198b9a-ca68-4b61-b4aa-40ce9badb88c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:44.555357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-17198b9a-ca68-4b61-b4aa-40ce9badb88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:44.596779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] The token cache is being invalidate because project 99606e335add4c5bb3573651418a298a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:44.597077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.597779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8b958e8a-3786-43a0-854a-96f93dce044b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:44.599392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 914/1812] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:44 2022] DELETE /identity/v3/projects/99606e335add4c5bb3573651418a298a => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:44.609721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5374f3d-7ac3-4ba1-9212-0df45734abb1 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:44.610014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5374f3d-7ac3-4ba1-9212-0df45734abb1 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:44.610297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5374f3d-7ac3-4ba1-9212-0df45734abb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:44.618389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 899/1813] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:44.628078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17198b9a-ca68-4b61-b4aa-40ce9badb88c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:44.629449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-17198b9a-ca68-4b61-b4aa-40ce9badb88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:44.629587 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e5374f3d-7ac3-4ba1-9212-0df45734abb1 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:44.630809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e5374f3d-7ac3-4ba1-9212-0df45734abb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:44.639459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 915/1814] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:44.648328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a971291-49ba-4b59-9742-e9202b4fe1af None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:44.648734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a971291-49ba-4b59-9742-e9202b4fe1af None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:44.649025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a971291-49ba-4b59-9742-e9202b4fe1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:44.666784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-9a971291-49ba-4b59-9742-e9202b4fe1af None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:44.668060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9a971291-49ba-4b59-9742-e9202b4fe1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:44.676466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 916/1815] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:44.681063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a971291-49ba-4b59-9742-e9202b4fe1af None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:44.688322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17198b9a-ca68-4b61-b4aa-40ce9badb88c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:44.690600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a971291-49ba-4b59-9742-e9202b4fe1af None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:44.691981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f644eec-f65c-4681-9edd-55d9d90b2caa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:44.692662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f644eec-f65c-4681-9edd-55d9d90b2caa service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:44.692952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f644eec-f65c-4681-9edd-55d9d90b2caa service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:44.697865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f644eec-f65c-4681-9edd-55d9d90b2caa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:44.697997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa77a09-96c0-40f3-95ac-93463ea8f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:44.697997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa77a09-96c0-40f3-95ac-93463ea8f3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:44.697997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa77a09-96c0-40f3-95ac-93463ea8f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:44.697997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa77a09-96c0-40f3-95ac-93463ea8f3f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:44.698322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa77a09-96c0-40f3-95ac-93463ea8f3f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:44.698403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa77a09-96c0-40f3-95ac-93463ea8f3f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:44.699957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa77a09-96c0-40f3-95ac-93463ea8f3f4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:44.703551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f644eec-f65c-4681-9edd-55d9d90b2caa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:44.705238 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f644eec-f65c-4681-9edd-55d9d90b2caa service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:44.712820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 917/1816] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:01:44.716098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 900/1817] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:01:44.716925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa77a09-96c0-40f3-95ac-93463ea8f3f4 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:44.729059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa77a09-96c0-40f3-95ac-93463ea8f3f4 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:44.734289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37254df5-017e-4f5a-862e-a967a9946962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:44.734961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37254df5-017e-4f5a-862e-a967a9946962 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:44.735250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37254df5-017e-4f5a-862e-a967a9946962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:44.735531 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37254df5-017e-4f5a-862e-a967a9946962 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:44.736268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37254df5-017e-4f5a-862e-a967a9946962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:44.738294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37254df5-017e-4f5a-862e-a967a9946962 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:44.836960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 918/1818] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:44 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:44.841185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f644eec-f65c-4681-9edd-55d9d90b2caa service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:44.851561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f644eec-f65c-4681-9edd-55d9d90b2caa service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:44.913789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dcf09-f505-4688-a831-771325baf942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:44.916078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dcf09-f505-4688-a831-771325baf942 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:44.916078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dcf09-f505-4688-a831-771325baf942 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:44.916333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dcf09-f505-4688-a831-771325baf942 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:44.917631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dcf09-f505-4688-a831-771325baf942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:44.920468 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dcf09-f505-4688-a831-771325baf942 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:44.934533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:45.000931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 901/1819] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:44 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:45.002004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37254df5-017e-4f5a-862e-a967a9946962 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.017733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37254df5-017e-4f5a-862e-a967a9946962 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.020301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b2410-fcaf-4eca-aafb-0d4d80a07a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.021049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b2410-fcaf-4eca-aafb-0d4d80a07a2f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.022193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b2410-fcaf-4eca-aafb-0d4d80a07a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.022193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b2410-fcaf-4eca-aafb-0d4d80a07a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.022846 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b2410-fcaf-4eca-aafb-0d4d80a07a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.024941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b2410-fcaf-4eca-aafb-0d4d80a07a2f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.041005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 919/1820] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:45.044747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dcf09-f505-4688-a831-771325baf942 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.058722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dcf09-f505-4688-a831-771325baf942 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.065015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216fa98-aa41-4b4e-91f5-44d838ea93b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.066046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216fa98-aa41-4b4e-91f5-44d838ea93b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.069113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216fa98-aa41-4b4e-91f5-44d838ea93b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.069113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216fa98-aa41-4b4e-91f5-44d838ea93b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.069113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216fa98-aa41-4b4e-91f5-44d838ea93b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.070252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216fa98-aa41-4b4e-91f5-44d838ea93b9 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.084682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 902/1821] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:45.086903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b2410-fcaf-4eca-aafb-0d4d80a07a2f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.097331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b2410-fcaf-4eca-aafb-0d4d80a07a2f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.103350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f74351-2c2d-4484-975b-e9007a9ac0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.104262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f74351-2c2d-4484-975b-e9007a9ac0f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.104551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f74351-2c2d-4484-975b-e9007a9ac0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.104869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f74351-2c2d-4484-975b-e9007a9ac0f6 admin admin] PATH_INFO: `/v3/projects/0a02698acc1e47d59f041fede0fd2ff3/users/84feedbbc91647aabbd4c6e71051079c/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.121579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f74351-2c2d-4484-975b-e9007a9ac0f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a02698acc1e47d59f041fede0fd2ff3, user_id=84feedbbc91647aabbd4c6e71051079c, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.128875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f74351-2c2d-4484-975b-e9007a9ac0f6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.204791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 920/1822] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:45 2022] PUT /identity/v3/projects/0a02698acc1e47d59f041fede0fd2ff3/users/84feedbbc91647aabbd4c6e71051079c/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:45.220510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd238aa-fd83-4e21-898a-e6f9abab3801 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.222687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd238aa-fd83-4e21-898a-e6f9abab3801 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.223094 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd238aa-fd83-4e21-898a-e6f9abab3801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.254386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6dd238aa-fd83-4e21-898a-e6f9abab3801 None None] MFA Rules not processed for user `84feedbbc91647aabbd4c6e71051079c`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:45.255613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6dd238aa-fd83-4e21-898a-e6f9abab3801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:45.317513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f74351-2c2d-4484-975b-e9007a9ac0f6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.319824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c9f74351-2c2d-4484-975b-e9007a9ac0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:45.339944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 903/1823] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:45.345874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd238aa-fd83-4e21-898a-e6f9abab3801 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.403373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f74351-2c2d-4484-975b-e9007a9ac0f6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.408432 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.409369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.409683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.409958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] PATH_INFO: `/v3/projects/cb774a313fd84e148308ed9fcf1a2512` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.411206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb774a313fd84e148308ed9fcf1a2512)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.414040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.418327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd238aa-fd83-4e21-898a-e6f9abab3801 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.423510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eade986-ce7a-46c3-9bd9-7dd32eb0f5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.424378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eade986-ce7a-46c3-9bd9-7dd32eb0f5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.424619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eade986-ce7a-46c3-9bd9-7dd32eb0f5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.424942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eade986-ce7a-46c3-9bd9-7dd32eb0f5d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.425644 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eade986-ce7a-46c3-9bd9-7dd32eb0f5d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.427562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eade986-ce7a-46c3-9bd9-7dd32eb0f5d8 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.516948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 904/1824] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:45 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:45.520897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eade986-ce7a-46c3-9bd9-7dd32eb0f5d8 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.527412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb774a313fd84e148308ed9fcf1a2512', 'request_id': 'req-71565d87-c79a-4f53-87fe-78a0cd1ae018'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:45.533237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb774a313fd84e148308ed9fcf1a2512', 'request_id': 'req-71565d87-c79a-4f53-87fe-78a0cd1ae018'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:45.540064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb774a313fd84e148308ed9fcf1a2512', 'request_id': 'req-71565d87-c79a-4f53-87fe-78a0cd1ae018'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:45.574554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:45.582949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eade986-ce7a-46c3-9bd9-7dd32eb0f5d8 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.588314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572ebee-851e-481a-846c-2423fa09c083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.589008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572ebee-851e-481a-846c-2423fa09c083 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.589292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572ebee-851e-481a-846c-2423fa09c083 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.589568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572ebee-851e-481a-846c-2423fa09c083 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.590405 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572ebee-851e-481a-846c-2423fa09c083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.592316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572ebee-851e-481a-846c-2423fa09c083 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.602311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] The token cache is being invalidate because project cb774a313fd84e148308ed9fcf1a2512 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:45.603780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:45.603780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:45.604850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 921/1825] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:45 2022] DELETE /identity/v3/projects/cb774a313fd84e148308ed9fcf1a2512 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:45.608852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.610212 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:45.674419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71565d87-c79a-4f53-87fe-78a0cd1ae018 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.679962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.680883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.681250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.681541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] PATH_INFO: `/v3/users/4e09a0c721094a788b54badccc64ea25` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.682756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e09a0c721094a788b54badccc64ea25)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.683870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 905/1826] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:45 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:45.685301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.685487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572ebee-851e-481a-846c-2423fa09c083 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.687150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4572ebee-851e-481a-846c-2423fa09c083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:45.702504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572ebee-851e-481a-846c-2423fa09c083 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.707192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df5324-216a-431c-80d7-007628f481a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.707865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df5324-216a-431c-80d7-007628f481a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.708156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df5324-216a-431c-80d7-007628f481a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.708459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df5324-216a-431c-80d7-007628f481a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.709236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df5324-216a-431c-80d7-007628f481a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.710938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df5324-216a-431c-80d7-007628f481a3 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.721216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 906/1827] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:45.725818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df5324-216a-431c-80d7-007628f481a3 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.735648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df5324-216a-431c-80d7-007628f481a3 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.740303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a97ce5-a6f8-4ee7-9e93-2e3d30612e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.741860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a97ce5-a6f8-4ee7-9e93-2e3d30612e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.742145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a97ce5-a6f8-4ee7-9e93-2e3d30612e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.742420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a97ce5-a6f8-4ee7-9e93-2e3d30612e10 admin admin] PATH_INFO: `/v3/projects/449a47b98a5546d189029e1f3957cc72/users/ddc44ba4ed85460a8d35af02e1983ef7/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.757241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a97ce5-a6f8-4ee7-9e93-2e3d30612e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=449a47b98a5546d189029e1f3957cc72, user_id=ddc44ba4ed85460a8d35af02e1983ef7, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.759931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a97ce5-a6f8-4ee7-9e93-2e3d30612e10 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.801323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:45.865829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 907/1828] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:45 2022] PUT /identity/v3/projects/449a47b98a5546d189029e1f3957cc72/users/ddc44ba4ed85460a8d35af02e1983ef7/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:45.870129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a97ce5-a6f8-4ee7-9e93-2e3d30612e10 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.884835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e09a0c721094a788b54badccc64ea25'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:45.896764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e09a0c721094a788b54badccc64ea25'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:45.903564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e09a0c721094a788b54badccc64ea25'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:45.909460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e09a0c721094a788b54badccc64ea25'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:45.933265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a97ce5-a6f8-4ee7-9e93-2e3d30612e10 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.937894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36730717-730f-4e98-bd36-bfc0c8646a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.941788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36730717-730f-4e98-bd36-bfc0c8646a33 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.941788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36730717-730f-4e98-bd36-bfc0c8646a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.941788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36730717-730f-4e98-bd36-bfc0c8646a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.941788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36730717-730f-4e98-bd36-bfc0c8646a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.941788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36730717-730f-4e98-bd36-bfc0c8646a33 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.952241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 908/1829] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:45.956386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36730717-730f-4e98-bd36-bfc0c8646a33 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:45.966344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36730717-730f-4e98-bd36-bfc0c8646a33 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:45.971011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876daa4-6fb3-4672-89b4-8a4f554d2fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:45.971879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876daa4-6fb3-4672-89b4-8a4f554d2fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:45.972435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876daa4-6fb3-4672-89b4-8a4f554d2fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:45.972744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876daa4-6fb3-4672-89b4-8a4f554d2fb0 admin admin] PATH_INFO: `/v3/projects/449a47b98a5546d189029e1f3957cc72/users/ddc44ba4ed85460a8d35af02e1983ef7/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:45.974696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876daa4-6fb3-4672-89b4-8a4f554d2fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=449a47b98a5546d189029e1f3957cc72, user_id=ddc44ba4ed85460a8d35af02e1983ef7, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:45.978011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876daa4-6fb3-4672-89b4-8a4f554d2fb0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:45.988271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 909/1830] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:45 2022] PUT /identity/v3/projects/449a47b98a5546d189029e1f3957cc72/users/ddc44ba4ed85460a8d35af02e1983ef7/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:45.990697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e09a0c721094a788b54badccc64ea25'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:46.001189 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 922/1831] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:45 2022] DELETE /identity/v3/users/4e09a0c721094a788b54badccc64ea25 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:46.002966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b800b-a114-4d5e-902c-a741e8cd8309 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:46.002966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b800b-a114-4d5e-902c-a741e8cd8309 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:46.002966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b800b-a114-4d5e-902c-a741e8cd8309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:46.032903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ac2b800b-a114-4d5e-902c-a741e8cd8309 None None] MFA Rules not processed for user `ddc44ba4ed85460a8d35af02e1983ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:46.034335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ac2b800b-a114-4d5e-902c-a741e8cd8309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:46.121831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 910/1832] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:46.133082 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:46.134549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:46.204887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528bef3-e582-4d1c-970f-aa4e1b69e547 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:46.209041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40c599-eac6-4c11-9e19-ef856f840218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:46.210545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c599-eac6-4c11-9e19-ef856f840218 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:46.210727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c599-eac6-4c11-9e19-ef856f840218 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:46.211069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c599-eac6-4c11-9e19-ef856f840218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:46.221279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40c599-eac6-4c11-9e19-ef856f840218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:46.222831 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40c599-eac6-4c11-9e19-ef856f840218 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:46.234011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 923/1833] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:46.260784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:46.349910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:46.636187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b800b-a114-4d5e-902c-a741e8cd8309 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:46.637639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ac2b800b-a114-4d5e-902c-a741e8cd8309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:46.703192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b800b-a114-4d5e-902c-a741e8cd8309 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:46.708226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:46.709136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:46.709530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:46.709818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] PATH_INFO: `/v3/projects/ac050a1fca1848cbb0f3060de2c32888` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:46.711032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac050a1fca1848cbb0f3060de2c32888)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:46.713014 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:46.738207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:46.853581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac050a1fca1848cbb0f3060de2c32888', 'request_id': 'req-9df98a90-8be4-4902-8135-6d0f71d89303'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:46.860379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac050a1fca1848cbb0f3060de2c32888', 'request_id': 'req-9df98a90-8be4-4902-8135-6d0f71d89303'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:46.933820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac050a1fca1848cbb0f3060de2c32888', 'request_id': 'req-9df98a90-8be4-4902-8135-6d0f71d89303'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:46.945353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] The token cache is being invalidate because project ac050a1fca1848cbb0f3060de2c32888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:46.945640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:46.946275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9df98a90-8be4-4902-8135-6d0f71d89303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:46.947818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 911/1834] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:46 2022] DELETE /identity/v3/projects/ac050a1fca1848cbb0f3060de2c32888 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:46.954061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6d83c-33b0-4496-bffd-4430625f8180 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:46.954349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6d83c-33b0-4496-bffd-4430625f8180 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:46.954629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6d83c-33b0-4496-bffd-4430625f8180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:46.957081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:46.976886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-91f6d83c-33b0-4496-bffd-4430625f8180 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:46.978093 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-91f6d83c-33b0-4496-bffd-4430625f8180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:47.039573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 924/1835] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:47.048813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb917f6-c693-4223-a94f-714bf2822eab None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:47.049108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb917f6-c693-4223-a94f-714bf2822eab None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:47.049390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb917f6-c693-4223-a94f-714bf2822eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:47.068106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8cb917f6-c693-4223-a94f-714bf2822eab None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:47.069975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8cb917f6-c693-4223-a94f-714bf2822eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:47.082208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 912/1836] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:47.083412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6d83c-33b0-4496-bffd-4430625f8180 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:47.093884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6d83c-33b0-4496-bffd-4430625f8180 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:47.098796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b76ae2-90a1-402c-b45b-42b302a6c5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:47.099583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b76ae2-90a1-402c-b45b-42b302a6c5ba admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:47.099779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b76ae2-90a1-402c-b45b-42b302a6c5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:47.100061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b76ae2-90a1-402c-b45b-42b302a6c5ba admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:47.107049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b76ae2-90a1-402c-b45b-42b302a6c5ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:47.107049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b76ae2-90a1-402c-b45b-42b302a6c5ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:47.107049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b76ae2-90a1-402c-b45b-42b302a6c5ba admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:47.118291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 925/1837] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:01:47.124564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb917f6-c693-4223-a94f-714bf2822eab None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:47.134093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb917f6-c693-4223-a94f-714bf2822eab None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:47.138843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdc874-6e58-4aa4-b176-e07a2e928c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:47.139507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc874-6e58-4aa4-b176-e07a2e928c29 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:47.139785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc874-6e58-4aa4-b176-e07a2e928c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:47.140058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc874-6e58-4aa4-b176-e07a2e928c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:47.140822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdc874-6e58-4aa4-b176-e07a2e928c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:47.142718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdc874-6e58-4aa4-b176-e07a2e928c29 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:47.312143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 913/1838] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:47 2022] POST /identity/v3/projects => generated 369 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:47.315985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b76ae2-90a1-402c-b45b-42b302a6c5ba admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:47.333068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b76ae2-90a1-402c-b45b-42b302a6c5ba admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:47.396397 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:47.414456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca732790-13bc-480b-a216-f4f1615a7b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:47.415153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca732790-13bc-480b-a216-f4f1615a7b6d admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:47.415447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca732790-13bc-480b-a216-f4f1615a7b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:47.415730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca732790-13bc-480b-a216-f4f1615a7b6d admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:47.416449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca732790-13bc-480b-a216-f4f1615a7b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:47.418596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca732790-13bc-480b-a216-f4f1615a7b6d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:47.501252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:47.506618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 926/1839] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:47 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:47.511024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdc874-6e58-4aa4-b176-e07a2e928c29 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:47.522869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdc874-6e58-4aa4-b176-e07a2e928c29 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:47.538822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8db260-722a-42fb-849d-779a20220cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:47.538822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8db260-722a-42fb-849d-779a20220cbe admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:47.538822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8db260-722a-42fb-849d-779a20220cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:47.538822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8db260-722a-42fb-849d-779a20220cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:47.546127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8db260-722a-42fb-849d-779a20220cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:47.548677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8db260-722a-42fb-849d-779a20220cbe admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:47.563606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 914/1840] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:47.570333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca732790-13bc-480b-a216-f4f1615a7b6d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:47.582909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca732790-13bc-480b-a216-f4f1615a7b6d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:47.588635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f621e1-f2ba-4262-8d09-2b3da1780c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:47.589525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f621e1-f2ba-4262-8d09-2b3da1780c15 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:47.591121 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f621e1-f2ba-4262-8d09-2b3da1780c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:47.591562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f621e1-f2ba-4262-8d09-2b3da1780c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:47.592672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f621e1-f2ba-4262-8d09-2b3da1780c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:47.596674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f621e1-f2ba-4262-8d09-2b3da1780c15 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:47.615146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 927/1841] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:47.616338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8db260-722a-42fb-849d-779a20220cbe admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:47.635429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8db260-722a-42fb-849d-779a20220cbe admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:47.650529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d9fb9-4da0-4c4c-92e7-1b7e7cbc2f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:47.651866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d9fb9-4da0-4c4c-92e7-1b7e7cbc2f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:47.652172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d9fb9-4da0-4c4c-92e7-1b7e7cbc2f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:47.652454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d9fb9-4da0-4c4c-92e7-1b7e7cbc2f42 admin admin] PATH_INFO: `/v3/projects/d4ff6904678f4a49a91913587c5b6e3f/users/9f2eb949f1a24163833fb07b01215555/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:47.676224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d9fb9-4da0-4c4c-92e7-1b7e7cbc2f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ff6904678f4a49a91913587c5b6e3f, user_id=9f2eb949f1a24163833fb07b01215555, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:47.680192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d9fb9-4da0-4c4c-92e7-1b7e7cbc2f42 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:47.805793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 915/1842] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:47 2022] PUT /identity/v3/projects/d4ff6904678f4a49a91913587c5b6e3f/users/9f2eb949f1a24163833fb07b01215555/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:47.808927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27486c1-95e0-42ca-88af-9a76117f880e None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:47.809230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27486c1-95e0-42ca-88af-9a76117f880e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:47.809556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27486c1-95e0-42ca-88af-9a76117f880e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:47.846524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c27486c1-95e0-42ca-88af-9a76117f880e None None] MFA Rules not processed for user `9f2eb949f1a24163833fb07b01215555`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:47.847941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c27486c1-95e0-42ca-88af-9a76117f880e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:47.934561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 928/1843] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:47.946666 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d9fb9-4da0-4c4c-92e7-1b7e7cbc2f42 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:47.948455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e28d9fb9-4da0-4c4c-92e7-1b7e7cbc2f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:48.014115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d9fb9-4da0-4c4c-92e7-1b7e7cbc2f42 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:48.018309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62d266-b50e-462c-8cb6-036e133228e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca33ef49ba44832a7fb45b2317d211c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:48.018962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62d266-b50e-462c-8cb6-036e133228e4 service glance] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:48.019243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62d266-b50e-462c-8cb6-036e133228e4 service glance] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:48.019517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62d266-b50e-462c-8cb6-036e133228e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:48.029676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62d266-b50e-462c-8cb6-036e133228e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:48.031098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62d266-b50e-462c-8cb6-036e133228e4 service glance] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:48.044283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 916/1844] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:48.621419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:49.290692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:49.820521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:49.989609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27486c1-95e0-42ca-88af-9a76117f880e None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:49.991436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c27486c1-95e0-42ca-88af-9a76117f880e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:50.125323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27486c1-95e0-42ca-88af-9a76117f880e None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:50.132881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:50.133953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:50.134558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:50.135101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] PATH_INFO: `/v3/users/497a01e93baa420d845628174b8726ef` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:50.137072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=497a01e93baa420d845628174b8726ef)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:50.139392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:50.592766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497a01e93baa420d845628174b8726ef'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:50.606106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:50.611879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497a01e93baa420d845628174b8726ef'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:50.629566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497a01e93baa420d845628174b8726ef'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:50.629566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497a01e93baa420d845628174b8726ef'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:50.740092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497a01e93baa420d845628174b8726ef'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:50.756032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 929/1845] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:49 2022] DELETE /identity/v3/users/497a01e93baa420d845628174b8726ef => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:50.759626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c528a-bf2d-45de-9f30-408f78c96111 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:50.760026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c528a-bf2d-45de-9f30-408f78c96111 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:50.760593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c528a-bf2d-45de-9f30-408f78c96111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:50.795762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b41c528a-bf2d-45de-9f30-408f78c96111 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:50.805400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b41c528a-bf2d-45de-9f30-408f78c96111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:50.899597 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 917/1846] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:50.933809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:50.939437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:51.027535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fda19f-28b0-446d-89fa-4ee9bef14cd4 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:51.032963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db195699-b99b-4f1e-9d9c-9898badd0c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:51.036830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db195699-b99b-4f1e-9d9c-9898badd0c13 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:51.036830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db195699-b99b-4f1e-9d9c-9898badd0c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:51.036830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db195699-b99b-4f1e-9d9c-9898badd0c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:51.048757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db195699-b99b-4f1e-9d9c-9898badd0c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:51.051971 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db195699-b99b-4f1e-9d9c-9898badd0c13 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:51.085351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 930/1847] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:51.811542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c528a-bf2d-45de-9f30-408f78c96111 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:51.813085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b41c528a-bf2d-45de-9f30-408f78c96111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:51.838551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c528a-bf2d-45de-9f30-408f78c96111 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:51.853440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:51.857329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:51.857329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:51.857329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] PATH_INFO: `/v3/projects/d433f409f9b14f928953c72559e139cd` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:51.857329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d433f409f9b14f928953c72559e139cd)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:51.858105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:51.965402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db195699-b99b-4f1e-9d9c-9898badd0c13 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:51.967832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-db195699-b99b-4f1e-9d9c-9898badd0c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:51.986229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db195699-b99b-4f1e-9d9c-9898badd0c13 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:52.005469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:52.005469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:52.005469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:52.005962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] PATH_INFO: `/v3/users/84feedbbc91647aabbd4c6e71051079c` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:52.008620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84feedbbc91647aabbd4c6e71051079c)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:52.013308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:52.079847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd433f409f9b14f928953c72559e139cd', 'request_id': 'req-2b613491-3ec1-4ba2-b7e6-1f81798311eb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.092966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd433f409f9b14f928953c72559e139cd', 'request_id': 'req-2b613491-3ec1-4ba2-b7e6-1f81798311eb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.198158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd433f409f9b14f928953c72559e139cd', 'request_id': 'req-2b613491-3ec1-4ba2-b7e6-1f81798311eb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.219534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] The token cache is being invalidate because project d433f409f9b14f928953c72559e139cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:52.219892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.221043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.223765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 918/1848] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:51 2022] DELETE /identity/v3/projects/d433f409f9b14f928953c72559e139cd => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:52.229225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:52.231018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:52.281439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:52.308984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84feedbbc91647aabbd4c6e71051079c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.310723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b613491-3ec1-4ba2-b7e6-1f81798311eb admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:52.317957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:52.319991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:52.320822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:52.321017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] PATH_INFO: `/v3/users/d798a8cdd5c94339b327de6c161eb193` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:52.323124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d798a8cdd5c94339b327de6c161eb193)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:52.325542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:52.325668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84feedbbc91647aabbd4c6e71051079c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.334202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84feedbbc91647aabbd4c6e71051079c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.335630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84feedbbc91647aabbd4c6e71051079c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.427425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6891c7dc-90bf-4ab5-88cb-547b03785ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84feedbbc91647aabbd4c6e71051079c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.437198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 931/1849] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:51 2022] DELETE /identity/v3/users/84feedbbc91647aabbd4c6e71051079c => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:52.442132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db12d5-1dcd-46fa-a7f1-b5c21d3217cd None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:52.442521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db12d5-1dcd-46fa-a7f1-b5c21d3217cd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:52.442876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db12d5-1dcd-46fa-a7f1-b5c21d3217cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:52.462867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c0db12d5-1dcd-46fa-a7f1-b5c21d3217cd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:52.464793 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c0db12d5-1dcd-46fa-a7f1-b5c21d3217cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:52.528184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:52.564276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 932/1850] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:52.574318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db12d5-1dcd-46fa-a7f1-b5c21d3217cd None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:52.580473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c0db12d5-1dcd-46fa-a7f1-b5c21d3217cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:52.581743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd798a8cdd5c94339b327de6c161eb193'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.591395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd798a8cdd5c94339b327de6c161eb193'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.592451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd798a8cdd5c94339b327de6c161eb193'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.669413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd798a8cdd5c94339b327de6c161eb193'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.688014 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd798a8cdd5c94339b327de6c161eb193'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:52.693235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db12d5-1dcd-46fa-a7f1-b5c21d3217cd None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:52.694822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 919/1851] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:52 2022] DELETE /identity/v3/users/d798a8cdd5c94339b327de6c161eb193 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:52.785187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:52.786266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:52.786754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:52.787465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:52.790779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:52.900728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:52.904918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:52.918837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 933/1852] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:53.324148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:53.332465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:53.353565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d7327-2f76-4bee-8ba5-bf52c9c0056c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:53.358873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:53.359946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:53.360851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:53.361134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] PATH_INFO: `/v3/users/793472a527614ba0905c0411d56056f0` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:53.366903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793472a527614ba0905c0411d56056f0)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:53.366903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:53.450446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:53.454493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:53.482597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3debc8-2426-433c-9aa6-586bd983c35d service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:53.503416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:53.505047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:53.516024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:53.516024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] PATH_INFO: `/v3/projects/3dcd9a9f615b41cfb5decdd8aad9b4b3` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:53.516024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dcd9a9f615b41cfb5decdd8aad9b4b3)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:53.516024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:53.647250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793472a527614ba0905c0411d56056f0'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.656150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793472a527614ba0905c0411d56056f0'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.656947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793472a527614ba0905c0411d56056f0'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.701384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dcd9a9f615b41cfb5decdd8aad9b4b3', 'request_id': 'req-49a58082-d90f-4695-a704-93451dc72fea'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.705481 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:53.710889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793472a527614ba0905c0411d56056f0'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.711010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dcd9a9f615b41cfb5decdd8aad9b4b3', 'request_id': 'req-49a58082-d90f-4695-a704-93451dc72fea'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.719893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dcd9a9f615b41cfb5decdd8aad9b4b3', 'request_id': 'req-49a58082-d90f-4695-a704-93451dc72fea'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.730665 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793472a527614ba0905c0411d56056f0'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.738124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 920/1853] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:53 2022] DELETE /identity/v3/users/793472a527614ba0905c0411d56056f0 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:53.739027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:53.740569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:53.791246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:53.797534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] The token cache is being invalidate because project 3dcd9a9f615b41cfb5decdd8aad9b4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:53.797816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.798663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-49a58082-d90f-4695-a704-93451dc72fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.800252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 934/1854] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:53 2022] DELETE /identity/v3/projects/3dcd9a9f615b41cfb5decdd8aad9b4b3 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:53.802748 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d0ce3-dc7b-4956-9bd0-467cac2409a6 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:53.803790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d0ce3-dc7b-4956-9bd0-467cac2409a6 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:53.804425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d0ce3-dc7b-4956-9bd0-467cac2409a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:53.823875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f39d0ce3-dc7b-4956-9bd0-467cac2409a6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:53.825195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f39d0ce3-dc7b-4956-9bd0-467cac2409a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:53.830863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1d516-d2ec-4f8e-89fd-f1a1a0fd2fb0 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:53.833245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 935/1855] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:53.836429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff63b3-cb91-4316-bd1e-922abef87b8e None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:53.836429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff63b3-cb91-4316-bd1e-922abef87b8e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:53.836429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff63b3-cb91-4316-bd1e-922abef87b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:53.839437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:53.840271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:53.840561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:53.840869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] PATH_INFO: `/v3/projects/0a02698acc1e47d59f041fede0fd2ff3` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:53.842092 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a02698acc1e47d59f041fede0fd2ff3)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:53.845686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:53.855184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a5ff63b3-cb91-4316-bd1e-922abef87b8e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:53.859324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a5ff63b3-cb91-4316-bd1e-922abef87b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:53.865320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 936/1856] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:53.867194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff63b3-cb91-4316-bd1e-922abef87b8e None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:53.868766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a5ff63b3-cb91-4316-bd1e-922abef87b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:53.937069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff63b3-cb91-4316-bd1e-922abef87b8e None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:53.951260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a02698acc1e47d59f041fede0fd2ff3', 'request_id': 'req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:53.958002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a02698acc1e47d59f041fede0fd2ff3', 'request_id': 'req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:54.008729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d5a79-112e-4d98-b721-c177689af342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:54.009402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d5a79-112e-4d98-b721-c177689af342 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.009683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d5a79-112e-4d98-b721-c177689af342 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.009963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d5a79-112e-4d98-b721-c177689af342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.013801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a02698acc1e47d59f041fede0fd2ff3', 'request_id': 'req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:54.026334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] The token cache is being invalidate because project 0a02698acc1e47d59f041fede0fd2ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:54.026647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:54.027317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9be9653b-7bf9-418c-8aa7-3c57ba10e1da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:54.028930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 921/1857] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:53 2022] DELETE /identity/v3/projects/0a02698acc1e47d59f041fede0fd2ff3 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:54.030238 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852d1e4-6d5b-43d1-9f2e-5e0598aeeb9e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.030531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852d1e4-6d5b-43d1-9f2e-5e0598aeeb9e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.030889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852d1e4-6d5b-43d1-9f2e-5e0598aeeb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.056758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6852d1e4-6d5b-43d1-9f2e-5e0598aeeb9e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:54.058045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6852d1e4-6d5b-43d1-9f2e-5e0598aeeb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:54.085825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d5a79-112e-4d98-b721-c177689af342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:54.087288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d5a79-112e-4d98-b721-c177689af342 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:54.098339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 937/1858] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:54.099033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d5a79-112e-4d98-b721-c177689af342 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:54.101065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-277d5a79-112e-4d98-b721-c177689af342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:54.114334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d5a79-112e-4d98-b721-c177689af342 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:54.119988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:54.120940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.127548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.127548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] PATH_INFO: `/v3/users/ddc44ba4ed85460a8d35af02e1983ef7` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.127548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddc44ba4ed85460a8d35af02e1983ef7)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:54.127548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:54.173894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 922/1859] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:54.180598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852d1e4-6d5b-43d1-9f2e-5e0598aeeb9e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:54.197192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852d1e4-6d5b-43d1-9f2e-5e0598aeeb9e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:54.202401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8f8c1-e93e-410b-92ed-db5a646a6eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:54.204544 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8f8c1-e93e-410b-92ed-db5a646a6eaa admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.204871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8f8c1-e93e-410b-92ed-db5a646a6eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.205503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8f8c1-e93e-410b-92ed-db5a646a6eaa admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.206366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8f8c1-e93e-410b-92ed-db5a646a6eaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:54.206685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8f8c1-e93e-410b-92ed-db5a646a6eaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:54.211662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8f8c1-e93e-410b-92ed-db5a646a6eaa admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:54.241416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 923/1860] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:01:54.245408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8f8c1-e93e-410b-92ed-db5a646a6eaa admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:54.263562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8f8c1-e93e-410b-92ed-db5a646a6eaa admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:54.274195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d6cd9-b044-46b1-8b8a-bb019fb5cc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:54.276362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d6cd9-b044-46b1-8b8a-bb019fb5cc97 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.276603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d6cd9-b044-46b1-8b8a-bb019fb5cc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.276913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d6cd9-b044-46b1-8b8a-bb019fb5cc97 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.277770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d6cd9-b044-46b1-8b8a-bb019fb5cc97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:54.280126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d6cd9-b044-46b1-8b8a-bb019fb5cc97 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:54.316020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:54.388847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddc44ba4ed85460a8d35af02e1983ef7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:54.414388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 924/1861] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:54 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:54.418326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d6cd9-b044-46b1-8b8a-bb019fb5cc97 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:54.439643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddc44ba4ed85460a8d35af02e1983ef7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:54.452503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddc44ba4ed85460a8d35af02e1983ef7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:54.452503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddc44ba4ed85460a8d35af02e1983ef7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:54.519542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d6cd9-b044-46b1-8b8a-bb019fb5cc97 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:54.527757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddc44ba4ed85460a8d35af02e1983ef7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:54.532101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45626b-b878-4c3c-bfa3-b929f6fbb40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:54.533041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45626b-b878-4c3c-bfa3-b929f6fbb40d admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.537604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45626b-b878-4c3c-bfa3-b929f6fbb40d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.537604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45626b-b878-4c3c-bfa3-b929f6fbb40d admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.537604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45626b-b878-4c3c-bfa3-b929f6fbb40d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:54.537604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45626b-b878-4c3c-bfa3-b929f6fbb40d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:54.542137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 938/1862] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:54 2022] DELETE /identity/v3/users/ddc44ba4ed85460a8d35af02e1983ef7 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:54.557849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:54.664689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 925/1863] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:54 2022] POST /identity/v3/users => generated 283 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:54.671164 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:54.673197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:54.835436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30195120-e633-4f45-9805-5047ed4607ed admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:54.846941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01573646-be93-4762-813b-1181e554fa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:54.849221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01573646-be93-4762-813b-1181e554fa71 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.849221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01573646-be93-4762-813b-1181e554fa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.849443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01573646-be93-4762-813b-1181e554fa71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.851410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01573646-be93-4762-813b-1181e554fa71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:54.854206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01573646-be93-4762-813b-1181e554fa71 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:54.866845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 939/1864] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:54.872787 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45626b-b878-4c3c-bfa3-b929f6fbb40d admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:54.889058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45626b-b878-4c3c-bfa3-b929f6fbb40d admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:54.894917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4963d9e-fc31-420b-973b-7d2c51c40636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:54.895866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4963d9e-fc31-420b-973b-7d2c51c40636 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.896228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4963d9e-fc31-420b-973b-7d2c51c40636 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.896579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4963d9e-fc31-420b-973b-7d2c51c40636 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.897450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4963d9e-fc31-420b-973b-7d2c51c40636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:54.899153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4963d9e-fc31-420b-973b-7d2c51c40636 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:54.910355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 926/1865] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:54.915229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01573646-be93-4762-813b-1181e554fa71 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:54.936348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4963d9e-fc31-420b-973b-7d2c51c40636 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:54.938320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01573646-be93-4762-813b-1181e554fa71 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:54.944743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c4963d9e-fc31-420b-973b-7d2c51c40636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:54.952010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55281f-f921-42f6-a54b-534f9ba14a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:54.954154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55281f-f921-42f6-a54b-534f9ba14a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.954602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55281f-f921-42f6-a54b-534f9ba14a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.963311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55281f-f921-42f6-a54b-534f9ba14a24 admin admin] PATH_INFO: `/v3/projects/99136a2dbbfc49a2bccce9d17eb5d6a6/users/ca2d681ab7274bb9b7e403fb05ad01c5/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.965718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4963d9e-fc31-420b-973b-7d2c51c40636 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:54.973421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:54.974661 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:54.975967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:54.977208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] PATH_INFO: `/v3/projects/54d28b408f5e445f98ae90382124f24d` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:54.978840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] RBAC: Authorizing `identity:delete_project(project_id=54d28b408f5e445f98ae90382124f24d)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:54.981931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:54.996924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55281f-f921-42f6-a54b-534f9ba14a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99136a2dbbfc49a2bccce9d17eb5d6a6, user_id=ca2d681ab7274bb9b7e403fb05ad01c5, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:55.002972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55281f-f921-42f6-a54b-534f9ba14a24 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:55.086105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 940/1866] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:54 2022] PUT /identity/v3/projects/99136a2dbbfc49a2bccce9d17eb5d6a6/users/ca2d681ab7274bb9b7e403fb05ad01c5/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:55.087441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a719847-4b3f-4834-b4fa-e5c24740e12d None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:55.089549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a719847-4b3f-4834-b4fa-e5c24740e12d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:55.089549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a719847-4b3f-4834-b4fa-e5c24740e12d None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:55.089984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 941/1867] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:01:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:01:55.098381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef4d16-db19-4954-bfc7-d386f2ba26c3 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:55.098883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef4d16-db19-4954-bfc7-d386f2ba26c3 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:55.099169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef4d16-db19-4954-bfc7-d386f2ba26c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:55.158910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-00ef4d16-db19-4954-bfc7-d386f2ba26c3 None None] MFA Rules not processed for user `ca2d681ab7274bb9b7e403fb05ad01c5`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:55.160576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-00ef4d16-db19-4954-bfc7-d386f2ba26c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:55.167065 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54d28b408f5e445f98ae90382124f24d', 'request_id': 'req-1883117d-0c14-4482-ba18-f5dba1a3559b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.176843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54d28b408f5e445f98ae90382124f24d', 'request_id': 'req-1883117d-0c14-4482-ba18-f5dba1a3559b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.236289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 942/1868] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:55.237743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ca868-a7f9-47b5-890d-c37372f8f4e5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:55.238037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ca868-a7f9-47b5-890d-c37372f8f4e5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:55.238321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ca868-a7f9-47b5-890d-c37372f8f4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:55.259097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:55.262442 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-910ca868-a7f9-47b5-890d-c37372f8f4e5 None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:55.262442 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-910ca868-a7f9-47b5-890d-c37372f8f4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:55.328422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54d28b408f5e445f98ae90382124f24d', 'request_id': 'req-1883117d-0c14-4482-ba18-f5dba1a3559b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.331235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 943/1869] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:01:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:55.332883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ca868-a7f9-47b5-890d-c37372f8f4e5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:55.344755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] The token cache is being invalidate because project 54d28b408f5e445f98ae90382124f24d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:55.345764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.346904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.348952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 927/1870] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:54 2022] DELETE /identity/v3/projects/54d28b408f5e445f98ae90382124f24d => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:55.349751 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:55.351600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:55.439685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ca868-a7f9-47b5-890d-c37372f8f4e5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:55.445009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5c564-1b2d-4046-901f-2d45cbfbdd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:55.445901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5c564-1b2d-4046-901f-2d45cbfbdd90 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:55.446260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5c564-1b2d-4046-901f-2d45cbfbdd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:55.446735 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5c564-1b2d-4046-901f-2d45cbfbdd90 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:55.447551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5c564-1b2d-4046-901f-2d45cbfbdd90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:55.449556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5c564-1b2d-4046-901f-2d45cbfbdd90 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:55.459528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883117d-0c14-4482-ba18-f5dba1a3559b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:55.464973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:55.466219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:55.466584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:55.466798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] PATH_INFO: `/v3/projects/449a47b98a5546d189029e1f3957cc72` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:55.468245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=449a47b98a5546d189029e1f3957cc72)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:55.470333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:55.576006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:55.579545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 944/1871] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:55 2022] POST /identity/v3/projects => generated 365 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:55.581127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5c564-1b2d-4046-901f-2d45cbfbdd90 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:55.583480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-17b5c564-1b2d-4046-901f-2d45cbfbdd90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:55.593436 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449a47b98a5546d189029e1f3957cc72', 'request_id': 'req-3a8c394d-a6e8-4195-aa85-10948ca89ad7'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.600723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449a47b98a5546d189029e1f3957cc72', 'request_id': 'req-3a8c394d-a6e8-4195-aa85-10948ca89ad7'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.669057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5c564-1b2d-4046-901f-2d45cbfbdd90 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:55.722896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449a47b98a5546d189029e1f3957cc72', 'request_id': 'req-3a8c394d-a6e8-4195-aa85-10948ca89ad7'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.739819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] The token cache is being invalidate because project 449a47b98a5546d189029e1f3957cc72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:55.740110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.741075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:55.742815 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 928/1872] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:55 2022] DELETE /identity/v3/projects/449a47b98a5546d189029e1f3957cc72 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:55.743801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:55.745818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:55.766384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:55.767503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:55.767885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:55.768236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:55.774474 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:55.793864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:55.795905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:55.812159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 945/1873] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:55.813157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:55.814409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:55.829658 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c394d-a6e8-4195-aa85-10948ca89ad7 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:55.833853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bdea7-0f0a-42ac-b44e-3f0f7f448b0f service nova] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:55.835793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:55.836965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:55.837334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:55.837676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] PATH_INFO: `/v3/users/526ce1eb4acd4fcca84e59abb0ab0bdd` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:55.840154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=526ce1eb4acd4fcca84e59abb0ab0bdd)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:55.843522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:55.853365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c629400-a232-49f2-bee7-4736d33fb290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:55.854070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c629400-a232-49f2-bee7-4736d33fb290 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:55.854465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c629400-a232-49f2-bee7-4736d33fb290 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:55.854756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c629400-a232-49f2-bee7-4736d33fb290 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:55.855468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c629400-a232-49f2-bee7-4736d33fb290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:55.877375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c629400-a232-49f2-bee7-4736d33fb290 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:55.995876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 946/1874] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:55 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:56.000131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcc8d3-0781-4062-aacb-043d64eecb32 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.000963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcc8d3-0781-4062-aacb-043d64eecb32 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.000963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcc8d3-0781-4062-aacb-043d64eecb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.035970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-b5bcc8d3-0781-4062-aacb-043d64eecb32 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:56.037843 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b5bcc8d3-0781-4062-aacb-043d64eecb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:56.064762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '526ce1eb4acd4fcca84e59abb0ab0bdd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:56.065597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:56.076421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '526ce1eb4acd4fcca84e59abb0ab0bdd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:56.077210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '526ce1eb4acd4fcca84e59abb0ab0bdd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:56.078035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 947/1875] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:56.079156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcc8d3-0781-4062-aacb-043d64eecb32 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.090760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcc8d3-0781-4062-aacb-043d64eecb32 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.096900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfd90b-ccab-41b1-9620-405e08bc47c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.100723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfd90b-ccab-41b1-9620-405e08bc47c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.101592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfd90b-ccab-41b1-9620-405e08bc47c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.102063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfd90b-ccab-41b1-9620-405e08bc47c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.103126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfd90b-ccab-41b1-9620-405e08bc47c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.105380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfd90b-ccab-41b1-9620-405e08bc47c4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:56.118887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 948/1876] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:56.121100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a4b91-781a-409c-a913-9f2e98e92bfd None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.121784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a4b91-781a-409c-a913-9f2e98e92bfd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.122767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a4b91-781a-409c-a913-9f2e98e92bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.153714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-bf4a4b91-781a-409c-a913-9f2e98e92bfd None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:56.158976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '526ce1eb4acd4fcca84e59abb0ab0bdd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:56.160430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-bf4a4b91-781a-409c-a913-9f2e98e92bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:56.179871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '526ce1eb4acd4fcca84e59abb0ab0bdd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:56.188234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 929/1877] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:55 2022] DELETE /identity/v3/users/526ce1eb4acd4fcca84e59abb0ab0bdd => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:56.188234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.191953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 949/1878] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:56.197424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:56.200408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a4b91-781a-409c-a913-9f2e98e92bfd None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.216392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a4b91-781a-409c-a913-9f2e98e92bfd None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.334272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:56.343690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a88940-6e13-4e59-9d41-f565a14e8767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.344664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a88940-6e13-4e59-9d41-f565a14e8767 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.345032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a88940-6e13-4e59-9d41-f565a14e8767 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.345721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a88940-6e13-4e59-9d41-f565a14e8767 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.346777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a88940-6e13-4e59-9d41-f565a14e8767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:56.347157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a88940-6e13-4e59-9d41-f565a14e8767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.349281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a88940-6e13-4e59-9d41-f565a14e8767 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:56.352158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bd297-1b63-49f0-97a5-f1b4d3c4ba62 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.357277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef820f2-a785-4dc7-bf6b-919f96cd38ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.358212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef820f2-a785-4dc7-bf6b-919f96cd38ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.358501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef820f2-a785-4dc7-bf6b-919f96cd38ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.358787 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef820f2-a785-4dc7-bf6b-919f96cd38ee admin admin] PATH_INFO: `/v3/projects/6c19c512fb284410ab35ae2f4e3cc0b4/users/412b0fa191d24b2f82b7299c4c289c66/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.363307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 950/1879] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:01:56.368134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a88940-6e13-4e59-9d41-f565a14e8767 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.379181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a88940-6e13-4e59-9d41-f565a14e8767 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.380429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef820f2-a785-4dc7-bf6b-919f96cd38ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c19c512fb284410ab35ae2f4e3cc0b4, user_id=412b0fa191d24b2f82b7299c4c289c66, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.385457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef820f2-a785-4dc7-bf6b-919f96cd38ee admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:56.391841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962464e5-d0e3-48c9-b57d-37f6eb96373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.392595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962464e5-d0e3-48c9-b57d-37f6eb96373c admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.392982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962464e5-d0e3-48c9-b57d-37f6eb96373c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.393325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962464e5-d0e3-48c9-b57d-37f6eb96373c admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.394096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962464e5-d0e3-48c9-b57d-37f6eb96373c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.396235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962464e5-d0e3-48c9-b57d-37f6eb96373c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:56.484906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 930/1880] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:56 2022] PUT /identity/v3/projects/6c19c512fb284410ab35ae2f4e3cc0b4/users/412b0fa191d24b2f82b7299c4c289c66/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:56.488665 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef820f2-a785-4dc7-bf6b-919f96cd38ee admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.491373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 951/1881] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:56 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:56.492605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962464e5-d0e3-48c9-b57d-37f6eb96373c admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.601138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef820f2-a785-4dc7-bf6b-919f96cd38ee admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.612796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:56.634615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962464e5-d0e3-48c9-b57d-37f6eb96373c admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.640139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d96d8-1c61-4d03-9e86-830ff326149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.641221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d96d8-1c61-4d03-9e86-830ff326149a admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.641609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d96d8-1c61-4d03-9e86-830ff326149a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.642069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d96d8-1c61-4d03-9e86-830ff326149a admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.643555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d96d8-1c61-4d03-9e86-830ff326149a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.646102 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d96d8-1c61-4d03-9e86-830ff326149a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:56.676118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5805920-e743-4c84-9ce4-0ec54e23840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.676967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5805920-e743-4c84-9ce4-0ec54e23840c admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.677356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5805920-e743-4c84-9ce4-0ec54e23840c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.677699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5805920-e743-4c84-9ce4-0ec54e23840c admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.678569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5805920-e743-4c84-9ce4-0ec54e23840c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.680370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5805920-e743-4c84-9ce4-0ec54e23840c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:56.691173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 931/1882] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:56.695242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5805920-e743-4c84-9ce4-0ec54e23840c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.705230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5805920-e743-4c84-9ce4-0ec54e23840c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.710993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de129d-0f4d-4bab-8d4f-4f6c8abf621f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.711906 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de129d-0f4d-4bab-8d4f-4f6c8abf621f admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.712190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de129d-0f4d-4bab-8d4f-4f6c8abf621f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.722218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de129d-0f4d-4bab-8d4f-4f6c8abf621f admin admin] PATH_INFO: `/v3/projects/6c19c512fb284410ab35ae2f4e3cc0b4/users/412b0fa191d24b2f82b7299c4c289c66/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.722218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de129d-0f4d-4bab-8d4f-4f6c8abf621f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c19c512fb284410ab35ae2f4e3cc0b4, user_id=412b0fa191d24b2f82b7299c4c289c66, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.722218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de129d-0f4d-4bab-8d4f-4f6c8abf621f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:56.730477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 932/1883] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:56 2022] PUT /identity/v3/projects/6c19c512fb284410ab35ae2f4e3cc0b4/users/412b0fa191d24b2f82b7299c4c289c66/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:56.743186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0cc8-d6e3-4f27-b275-0b39fb767d09 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.743964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0cc8-d6e3-4f27-b275-0b39fb767d09 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.744344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0cc8-d6e3-4f27-b275-0b39fb767d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.749682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 952/1884] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:56 2022] POST /identity/v3/users => generated 294 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:56.753508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d96d8-1c61-4d03-9e86-830ff326149a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.765491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d96d8-1c61-4d03-9e86-830ff326149a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.785758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b30b0cc8-d6e3-4f27-b275-0b39fb767d09 None None] MFA Rules not processed for user `412b0fa191d24b2f82b7299c4c289c66`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:56.788369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b30b0cc8-d6e3-4f27-b275-0b39fb767d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:56.845403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4865d-b9c4-4141-bf95-baf5b410686a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.846102 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4865d-b9c4-4141-bf95-baf5b410686a admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.846573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4865d-b9c4-4141-bf95-baf5b410686a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.846863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4865d-b9c4-4141-bf95-baf5b410686a admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.847661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4865d-b9c4-4141-bf95-baf5b410686a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.849488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4865d-b9c4-4141-bf95-baf5b410686a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:56.860818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 953/1885] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:56.864383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4865d-b9c4-4141-bf95-baf5b410686a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.878091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4865d-b9c4-4141-bf95-baf5b410686a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.880868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 933/1886] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:56.882435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0cc8-d6e3-4f27-b275-0b39fb767d09 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.884561 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b30b0cc8-d6e3-4f27-b275-0b39fb767d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:56.887378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d346b-f612-4fe3-90da-f7f007adb409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.887983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d346b-f612-4fe3-90da-f7f007adb409 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.888350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d346b-f612-4fe3-90da-f7f007adb409 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.888535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d346b-f612-4fe3-90da-f7f007adb409 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.889960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d346b-f612-4fe3-90da-f7f007adb409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.890951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d346b-f612-4fe3-90da-f7f007adb409 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:56.903519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 954/1887] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:56.904400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d346b-f612-4fe3-90da-f7f007adb409 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:56.908322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-cc3d346b-f612-4fe3-90da-f7f007adb409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:56.913221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0cc8-d6e3-4f27-b275-0b39fb767d09 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:56.922399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:56.923905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:56.924190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:56.924467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] PATH_INFO: `/v3/projects/d35c4a2b642949658935d4da44b298e4` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:56.928255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d35c4a2b642949658935d4da44b298e4)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:56.928610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.004804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d346b-f612-4fe3-90da-f7f007adb409 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.009052 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed08e-2152-4074-bf62-788677e5c16c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.009708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed08e-2152-4074-bf62-788677e5c16c service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.010001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed08e-2152-4074-bf62-788677e5c16c service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.010288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed08e-2152-4074-bf62-788677e5c16c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.019403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fed08e-2152-4074-bf62-788677e5c16c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.020855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fed08e-2152-4074-bf62-788677e5c16c service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.030799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 955/1888] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:57.031560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed08e-2152-4074-bf62-788677e5c16c service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.041504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed08e-2152-4074-bf62-788677e5c16c service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.047570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b77487-ef7d-4cfc-83fc-d57cb18f85f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.051729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b77487-ef7d-4cfc-83fc-d57cb18f85f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.051729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b77487-ef7d-4cfc-83fc-d57cb18f85f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.051729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b77487-ef7d-4cfc-83fc-d57cb18f85f1 admin admin] PATH_INFO: `/v3/projects/01576616821d494885b743bb4cf6f6a7/users/ec02c25a679f4d34a06cff6b38b886e6/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.061417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd35c4a2b642949658935d4da44b298e4', 'request_id': 'req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:57.064663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b77487-ef7d-4cfc-83fc-d57cb18f85f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01576616821d494885b743bb4cf6f6a7, user_id=ec02c25a679f4d34a06cff6b38b886e6, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.067328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b77487-ef7d-4cfc-83fc-d57cb18f85f1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.067919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd35c4a2b642949658935d4da44b298e4', 'request_id': 'req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:57.166585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd35c4a2b642949658935d4da44b298e4', 'request_id': 'req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:57.176358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 956/1889] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:57 2022] PUT /identity/v3/projects/01576616821d494885b743bb4cf6f6a7/users/ec02c25a679f4d34a06cff6b38b886e6/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:57.193047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] The token cache is being invalidate because project d35c4a2b642949658935d4da44b298e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:01:57.193455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:57.194965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-50ebb25d-1dbf-4f6f-b5c7-eddfb1724198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:57.199688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72f8d6-d0de-4a9a-acc9-cbacd0bf3d40 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.199867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 934/1890] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:56 2022] DELETE /identity/v3/projects/d35c4a2b642949658935d4da44b298e4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:57.199950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72f8d6-d0de-4a9a-acc9-cbacd0bf3d40 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.199950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72f8d6-d0de-4a9a-acc9-cbacd0bf3d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.201918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e11e11-7756-4c33-92ff-6bafb3f86a16 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.202209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e11e11-7756-4c33-92ff-6bafb3f86a16 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.202491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e11e11-7756-4c33-92ff-6bafb3f86a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.222081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-01e11e11-7756-4c33-92ff-6bafb3f86a16 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:57.223565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-01e11e11-7756-4c33-92ff-6bafb3f86a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:57.249494 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-4e72f8d6-d0de-4a9a-acc9-cbacd0bf3d40 None None] MFA Rules not processed for user `ec02c25a679f4d34a06cff6b38b886e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:57.250743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4e72f8d6-d0de-4a9a-acc9-cbacd0bf3d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:57.321231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 935/1891] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:57.321886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e11e11-7756-4c33-92ff-6bafb3f86a16 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.323173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-01e11e11-7756-4c33-92ff-6bafb3f86a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:57.335356 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 957/1892] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:57.336790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f09b46-a621-4ab5-bfb2-35b63c8632b4 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.337093 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f09b46-a621-4ab5-bfb2-35b63c8632b4 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.337378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f09b46-a621-4ab5-bfb2-35b63c8632b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.357110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f2f09b46-a621-4ab5-bfb2-35b63c8632b4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:57.366484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:57.366484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f2f09b46-a621-4ab5-bfb2-35b63c8632b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:57.369424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 958/1893] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:57.370134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f09b46-a621-4ab5-bfb2-35b63c8632b4 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.373585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f2f09b46-a621-4ab5-bfb2-35b63c8632b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:57.404252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f09b46-a621-4ab5-bfb2-35b63c8632b4 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.417334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e11e11-7756-4c33-92ff-6bafb3f86a16 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.417884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2acf6-b5df-42ac-b111-09ac6b4dd2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.418889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2acf6-b5df-42ac-b111-09ac6b4dd2ca admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.419185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2acf6-b5df-42ac-b111-09ac6b4dd2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.419472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2acf6-b5df-42ac-b111-09ac6b4dd2ca admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.421264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2acf6-b5df-42ac-b111-09ac6b4dd2ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.421569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be24a33-8384-4205-b828-5925ef3f3548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.422242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be24a33-8384-4205-b828-5925ef3f3548 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.422531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be24a33-8384-4205-b828-5925ef3f3548 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.422810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be24a33-8384-4205-b828-5925ef3f3548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.423876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2acf6-b5df-42ac-b111-09ac6b4dd2ca admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.424697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1be24a33-8384-4205-b828-5925ef3f3548 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:57.464426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:57.496210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be24a33-8384-4205-b828-5925ef3f3548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.497893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be24a33-8384-4205-b828-5925ef3f3548 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.507614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 936/1894] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:57.508352 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be24a33-8384-4205-b828-5925ef3f3548 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.519536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be24a33-8384-4205-b828-5925ef3f3548 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.523202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 959/1895] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:57 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:57.524980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13fbce-6862-4af7-a02a-5686d596d7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.525654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13fbce-6862-4af7-a02a-5686d596d7db admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.526111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13fbce-6862-4af7-a02a-5686d596d7db admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.526220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13fbce-6862-4af7-a02a-5686d596d7db admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.526946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13fbce-6862-4af7-a02a-5686d596d7db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:01:57.527201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2acf6-b5df-42ac-b111-09ac6b4dd2ca admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.527325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13fbce-6862-4af7-a02a-5686d596d7db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.528870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13fbce-6862-4af7-a02a-5686d596d7db admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.547098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 937/1896] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:01:57.550875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13fbce-6862-4af7-a02a-5686d596d7db admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.573636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13fbce-6862-4af7-a02a-5686d596d7db admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.607226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2acf6-b5df-42ac-b111-09ac6b4dd2ca admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.612755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ae99-481e-4ca1-8123-5b6178f8fc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.616405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ae99-481e-4ca1-8123-5b6178f8fc5f admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.616405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ae99-481e-4ca1-8123-5b6178f8fc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.616405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ae99-481e-4ca1-8123-5b6178f8fc5f admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.616405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6ae99-481e-4ca1-8123-5b6178f8fc5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.617373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6ae99-481e-4ca1-8123-5b6178f8fc5f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.637996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eadfcc-01d6-468c-880f-3239bcb5b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.638655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eadfcc-01d6-468c-880f-3239bcb5b431 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.638946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eadfcc-01d6-468c-880f-3239bcb5b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.639232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eadfcc-01d6-468c-880f-3239bcb5b431 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.644378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eadfcc-01d6-468c-880f-3239bcb5b431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.644378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eadfcc-01d6-468c-880f-3239bcb5b431 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.734172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 960/1897] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:57 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:57.738576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ae99-481e-4ca1-8123-5b6178f8fc5f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.751070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ae99-481e-4ca1-8123-5b6178f8fc5f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.751273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 938/1898] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:57 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:57.755921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eadfcc-01d6-468c-880f-3239bcb5b431 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.814292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaefee-a004-4f89-abd0-cc7efee04427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.814960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaefee-a004-4f89-abd0-cc7efee04427 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.815247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaefee-a004-4f89-abd0-cc7efee04427 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.815524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaefee-a004-4f89-abd0-cc7efee04427 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.816254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adaefee-a004-4f89-abd0-cc7efee04427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.817935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adaefee-a004-4f89-abd0-cc7efee04427 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.821188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eadfcc-01d6-468c-880f-3239bcb5b431 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.826836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbaf28-2bf7-4cbb-9071-3597a05686cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.827671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbaf28-2bf7-4cbb-9071-3597a05686cf admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.828019 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbaf28-2bf7-4cbb-9071-3597a05686cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.831185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 961/1899] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:57.831318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbaf28-2bf7-4cbb-9071-3597a05686cf admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.831318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbaf28-2bf7-4cbb-9071-3597a05686cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.833678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbaf28-2bf7-4cbb-9071-3597a05686cf admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.833946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaefee-a004-4f89-abd0-cc7efee04427 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.843252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaefee-a004-4f89-abd0-cc7efee04427 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.848412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67dbf7-a62e-4bd0-a1eb-c93a963655bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.850675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67dbf7-a62e-4bd0-a1eb-c93a963655bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.850924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67dbf7-a62e-4bd0-a1eb-c93a963655bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.851194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67dbf7-a62e-4bd0-a1eb-c93a963655bb admin admin] PATH_INFO: `/v3/projects/533517fb139041fc8cdc568748aa83a2/users/42bc55ed63af46aab5b5eddd0cc93464/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.870859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67dbf7-a62e-4bd0-a1eb-c93a963655bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=533517fb139041fc8cdc568748aa83a2, user_id=42bc55ed63af46aab5b5eddd0cc93464, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.873633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67dbf7-a62e-4bd0-a1eb-c93a963655bb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.946418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 939/1900] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:57 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:57.950168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbaf28-2bf7-4cbb-9071-3597a05686cf admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:57.961484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbaf28-2bf7-4cbb-9071-3597a05686cf admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:57.966133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f894c-2a69-42fd-b384-472272334b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:57.966987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f894c-2a69-42fd-b384-472272334b45 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:57.968311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f894c-2a69-42fd-b384-472272334b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:57.968834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f894c-2a69-42fd-b384-472272334b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:57.969659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f894c-2a69-42fd-b384-472272334b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:57.971698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f894c-2a69-42fd-b384-472272334b45 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:57.978183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 962/1901] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:57 2022] PUT /identity/v3/projects/533517fb139041fc8cdc568748aa83a2/users/42bc55ed63af46aab5b5eddd0cc93464/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:57.996534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67dbf7-a62e-4bd0-a1eb-c93a963655bb admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.012218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 940/1902] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:58.016035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f894c-2a69-42fd-b384-472272334b45 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.030932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f894c-2a69-42fd-b384-472272334b45 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.051831 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:58.137516 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c5214-5052-4878-a3c7-88b5e46467c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.138602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c5214-5052-4878-a3c7-88b5e46467c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.139593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c5214-5052-4878-a3c7-88b5e46467c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.147692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:58.148312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c5214-5052-4878-a3c7-88b5e46467c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.149462 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c5214-5052-4878-a3c7-88b5e46467c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.151352 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c5214-5052-4878-a3c7-88b5e46467c9 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.162205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 941/1903] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:58.166775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67dbf7-a62e-4bd0-a1eb-c93a963655bb admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.176126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcae12-8f2f-4597-a7ea-3c5eb365c7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.176126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcae12-8f2f-4597-a7ea-3c5eb365c7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.176126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcae12-8f2f-4597-a7ea-3c5eb365c7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.176126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcae12-8f2f-4597-a7ea-3c5eb365c7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.177078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcae12-8f2f-4597-a7ea-3c5eb365c7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.177078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcae12-8f2f-4597-a7ea-3c5eb365c7d9 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.177574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c5214-5052-4878-a3c7-88b5e46467c9 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.188439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c5214-5052-4878-a3c7-88b5e46467c9 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.190636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 963/1904] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:01:58.195146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcae12-8f2f-4597-a7ea-3c5eb365c7d9 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.196928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbceb6-8683-4a15-924d-6c5ced966851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.198085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbceb6-8683-4a15-924d-6c5ced966851 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.198465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbceb6-8683-4a15-924d-6c5ced966851 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.198847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbceb6-8683-4a15-924d-6c5ced966851 admin admin] PATH_INFO: `/v3/projects/fbf47a92239b40b19bd4e1cf35ebdce8/users/975128f2a6d04e719e0643a32240d00b/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.207177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcae12-8f2f-4597-a7ea-3c5eb365c7d9 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.214407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000feab-d1ae-4197-a20b-18655fb85452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.215464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000feab-d1ae-4197-a20b-18655fb85452 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.215754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000feab-d1ae-4197-a20b-18655fb85452 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.216087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000feab-d1ae-4197-a20b-18655fb85452 admin admin] PATH_INFO: `/v3/projects/533517fb139041fc8cdc568748aa83a2/users/42bc55ed63af46aab5b5eddd0cc93464/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.218556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000feab-d1ae-4197-a20b-18655fb85452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533517fb139041fc8cdc568748aa83a2, user_id=42bc55ed63af46aab5b5eddd0cc93464, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.219006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbceb6-8683-4a15-924d-6c5ced966851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf47a92239b40b19bd4e1cf35ebdce8, user_id=975128f2a6d04e719e0643a32240d00b, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.221591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000feab-d1ae-4197-a20b-18655fb85452 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.222286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbceb6-8683-4a15-924d-6c5ced966851 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.235280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 964/1905] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:58 2022] PUT /identity/v3/projects/533517fb139041fc8cdc568748aa83a2/users/42bc55ed63af46aab5b5eddd0cc93464/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:58.250477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e451c6-c13d-489c-9be5-8e11fee5c341 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.250914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e451c6-c13d-489c-9be5-8e11fee5c341 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.251203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e451c6-c13d-489c-9be5-8e11fee5c341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.279801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-86e451c6-c13d-489c-9be5-8e11fee5c341 None None] MFA Rules not processed for user `42bc55ed63af46aab5b5eddd0cc93464`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:58.282153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-86e451c6-c13d-489c-9be5-8e11fee5c341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:58.304425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 942/1906] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:58 2022] PUT /identity/v3/projects/fbf47a92239b40b19bd4e1cf35ebdce8/users/975128f2a6d04e719e0643a32240d00b/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:58.315501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013eb2e-d135-43dc-9590-826b2ac47509 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.315952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013eb2e-d135-43dc-9590-826b2ac47509 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.316322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013eb2e-d135-43dc-9590-826b2ac47509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.355886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-2013eb2e-d135-43dc-9590-826b2ac47509 None None] MFA Rules not processed for user `975128f2a6d04e719e0643a32240d00b`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:58.357262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2013eb2e-d135-43dc-9590-826b2ac47509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:58.370981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 965/1907] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:58.383040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e451c6-c13d-489c-9be5-8e11fee5c341 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.464157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 943/1908] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:58.468680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013eb2e-d135-43dc-9590-826b2ac47509 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.494294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e451c6-c13d-489c-9be5-8e11fee5c341 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.559940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f248d-5914-46da-90c7-769fe1aa7e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.560622 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f248d-5914-46da-90c7-769fe1aa7e0f service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.561625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f248d-5914-46da-90c7-769fe1aa7e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.561804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f248d-5914-46da-90c7-769fe1aa7e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.596269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013eb2e-d135-43dc-9590-826b2ac47509 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.603273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c9436-3c5c-458a-bd38-f3dc690ddb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.603273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c9436-3c5c-458a-bd38-f3dc690ddb82 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.603273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c9436-3c5c-458a-bd38-f3dc690ddb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.603273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c9436-3c5c-458a-bd38-f3dc690ddb82 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.603877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:58.603967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c9436-3c5c-458a-bd38-f3dc690ddb82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.606632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c9436-3c5c-458a-bd38-f3dc690ddb82 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.659623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f248d-5914-46da-90c7-769fe1aa7e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.661319 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f248d-5914-46da-90c7-769fe1aa7e0f service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.675453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 966/1909] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:01:58.689351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 944/1910] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:01:58 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:01:58.693334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f248d-5914-46da-90c7-769fe1aa7e0f service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.706833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f248d-5914-46da-90c7-769fe1aa7e0f service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.710017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c9436-3c5c-458a-bd38-f3dc690ddb82 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.777387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af4dd3-0248-4dfb-a080-04c94d4ff448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.778267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af4dd3-0248-4dfb-a080-04c94d4ff448 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.778718 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af4dd3-0248-4dfb-a080-04c94d4ff448 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.779123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af4dd3-0248-4dfb-a080-04c94d4ff448 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.779741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c9436-3c5c-458a-bd38-f3dc690ddb82 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.780027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af4dd3-0248-4dfb-a080-04c94d4ff448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.782191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af4dd3-0248-4dfb-a080-04c94d4ff448 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.785632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.786525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.786837 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.787180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] PATH_INFO: `/v3/users/ec02c25a679f4d34a06cff6b38b886e6` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.788343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec02c25a679f4d34a06cff6b38b886e6)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.790501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.883340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 967/1911] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:01:58 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:01:58.887267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af4dd3-0248-4dfb-a080-04c94d4ff448 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.898420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af4dd3-0248-4dfb-a080-04c94d4ff448 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.903021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c1d84-0811-4f06-979e-df0546c0e284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.903690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c1d84-0811-4f06-979e-df0546c0e284 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.903974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c1d84-0811-4f06-979e-df0546c0e284 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.904253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c1d84-0811-4f06-979e-df0546c0e284 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.905019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c1d84-0811-4f06-979e-df0546c0e284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.906608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c1d84-0811-4f06-979e-df0546c0e284 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.916675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 968/1912] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:58.921010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c1d84-0811-4f06-979e-df0546c0e284 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:58.932897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c1d84-0811-4f06-979e-df0546c0e284 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:58.937673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a4ed3-d250-407e-8f66-9466953e47ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:58.938572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a4ed3-d250-407e-8f66-9466953e47ab admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:58.938856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a4ed3-d250-407e-8f66-9466953e47ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:58.939252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a4ed3-d250-407e-8f66-9466953e47ab admin admin] PATH_INFO: `/v3/projects/209bfbb387514f499868a78461fcd568/users/98ccd3cbb6dc4efbae634685b3227370/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:58.956539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a4ed3-d250-407e-8f66-9466953e47ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=209bfbb387514f499868a78461fcd568, user_id=98ccd3cbb6dc4efbae634685b3227370, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:58.959868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a4ed3-d250-407e-8f66-9466953e47ab admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:58.999154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:59.026140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec02c25a679f4d34a06cff6b38b886e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.042664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 969/1913] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:58 2022] PUT /identity/v3/projects/209bfbb387514f499868a78461fcd568/users/98ccd3cbb6dc4efbae634685b3227370/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:59.043739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a4ed3-d250-407e-8f66-9466953e47ab admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:59.046133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec02c25a679f4d34a06cff6b38b886e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.047026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec02c25a679f4d34a06cff6b38b886e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.120816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a4ed3-d250-407e-8f66-9466953e47ab admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:59.126332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a413548-b7a4-4cce-85fa-3baa56f1e018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:59.127842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a413548-b7a4-4cce-85fa-3baa56f1e018 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:59.127842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a413548-b7a4-4cce-85fa-3baa56f1e018 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:59.127842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a413548-b7a4-4cce-85fa-3baa56f1e018 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:59.128375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a413548-b7a4-4cce-85fa-3baa56f1e018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:59.130045 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a413548-b7a4-4cce-85fa-3baa56f1e018 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:59.140521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 970/1914] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:01:59.145514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec02c25a679f4d34a06cff6b38b886e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.145658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a413548-b7a4-4cce-85fa-3baa56f1e018 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:59.161599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a413548-b7a4-4cce-85fa-3baa56f1e018 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:59.161712 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d37b57cf-9aa4-4509-88bc-89236d8bd255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec02c25a679f4d34a06cff6b38b886e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.164224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796033-4696-4c7d-9475-03f90217b699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:59.165154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796033-4696-4c7d-9475-03f90217b699 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:59.165444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796033-4696-4c7d-9475-03f90217b699 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:59.165725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796033-4696-4c7d-9475-03f90217b699 admin admin] PATH_INFO: `/v3/projects/209bfbb387514f499868a78461fcd568/users/98ccd3cbb6dc4efbae634685b3227370/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:59.166895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 945/1915] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:58 2022] DELETE /identity/v3/users/ec02c25a679f4d34a06cff6b38b886e6 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:01:59.167903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44796033-4696-4c7d-9475-03f90217b699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209bfbb387514f499868a78461fcd568, user_id=98ccd3cbb6dc4efbae634685b3227370, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:59.170889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44796033-4696-4c7d-9475-03f90217b699 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:59.171591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b091d79-dcff-4fb0-8e9b-c71168f75b7c None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:59.171950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b091d79-dcff-4fb0-8e9b-c71168f75b7c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:59.172411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b091d79-dcff-4fb0-8e9b-c71168f75b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:59.182965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 971/1916] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:01:59 2022] PUT /identity/v3/projects/209bfbb387514f499868a78461fcd568/users/98ccd3cbb6dc4efbae634685b3227370/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:59.191433 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-2b091d79-dcff-4fb0-8e9b-c71168f75b7c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:59.192669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2b091d79-dcff-4fb0-8e9b-c71168f75b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:59.199407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069bc42-e725-4206-829d-6164fe5e17d9 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:59.199743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069bc42-e725-4206-829d-6164fe5e17d9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:59.199968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069bc42-e725-4206-829d-6164fe5e17d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:59.229334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-b069bc42-e725-4206-829d-6164fe5e17d9 None None] MFA Rules not processed for user `98ccd3cbb6dc4efbae634685b3227370`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:59.230613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b069bc42-e725-4206-829d-6164fe5e17d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:59.266530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 946/1917] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:01:59.276818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b091d79-dcff-4fb0-8e9b-c71168f75b7c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:59.280788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2b091d79-dcff-4fb0-8e9b-c71168f75b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:59.335804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 972/1918] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:01:59.342921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:59.352139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069bc42-e725-4206-829d-6164fe5e17d9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:59.356039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b069bc42-e725-4206-829d-6164fe5e17d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:59.374764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b091d79-dcff-4fb0-8e9b-c71168f75b7c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:59.380206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226d614-faec-40e9-a48e-fe63fedb5881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:59.384886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226d614-faec-40e9-a48e-fe63fedb5881 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:59.384886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226d614-faec-40e9-a48e-fe63fedb5881 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:59.384886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226d614-faec-40e9-a48e-fe63fedb5881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:59.421347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226d614-faec-40e9-a48e-fe63fedb5881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:59.423617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226d614-faec-40e9-a48e-fe63fedb5881 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:59.438466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 947/1919] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:59.453944 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069bc42-e725-4206-829d-6164fe5e17d9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:59.458511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57306e8c-c26b-48d5-83ba-63ce0701f20b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:59.462011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57306e8c-c26b-48d5-83ba-63ce0701f20b service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:59.462011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57306e8c-c26b-48d5-83ba-63ce0701f20b service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:59.462011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57306e8c-c26b-48d5-83ba-63ce0701f20b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:59.472787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57306e8c-c26b-48d5-83ba-63ce0701f20b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:59.474461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57306e8c-c26b-48d5-83ba-63ce0701f20b service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:59.487258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 973/1920] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:01:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:01:59.572239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226d614-faec-40e9-a48e-fe63fedb5881 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:59.574024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f226d614-faec-40e9-a48e-fe63fedb5881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:59.587977 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226d614-faec-40e9-a48e-fe63fedb5881 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:59.592235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:59.593154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:59.593442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:59.593722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] PATH_INFO: `/v3/users/975128f2a6d04e719e0643a32240d00b` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:59.594944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=975128f2a6d04e719e0643a32240d00b)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:59.596893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:59.794139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '975128f2a6d04e719e0643a32240d00b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.810515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '975128f2a6d04e719e0643a32240d00b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.812345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '975128f2a6d04e719e0643a32240d00b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.924092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57306e8c-c26b-48d5-83ba-63ce0701f20b service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:01:59.928027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '975128f2a6d04e719e0643a32240d00b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.928941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-57306e8c-c26b-48d5-83ba-63ce0701f20b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:01:59.932842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:01:59.952070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-76fac699-053f-4802-912d-4e891a2e2901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '975128f2a6d04e719e0643a32240d00b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:01:59.958721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57306e8c-c26b-48d5-83ba-63ce0701f20b service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:01:59.964523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 948/1921] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:01:59 2022] DELETE /identity/v3/users/975128f2a6d04e719e0643a32240d00b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:01:59.969790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4072a1-7ca8-420b-9092-04f3d05eb7c4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:59.970087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4072a1-7ca8-420b-9092-04f3d05eb7c4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:59.970717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4072a1-7ca8-420b-9092-04f3d05eb7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:59.977099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:01:59.978281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:01:59.979059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:01:59.979896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] PATH_INFO: `/v3/projects/01576616821d494885b743bb4cf6f6a7` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:01:59.981558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] RBAC: Authorizing `identity:delete_project(project_id=01576616821d494885b743bb4cf6f6a7)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:01:59.984952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:01:59.996446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8f4072a1-7ca8-420b-9092-04f3d05eb7c4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:01:59.997723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8f4072a1-7ca8-420b-9092-04f3d05eb7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:00.073712 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 949/1922] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:00.083208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4072a1-7ca8-420b-9092-04f3d05eb7c4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:00.085482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8f4072a1-7ca8-420b-9092-04f3d05eb7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:00.132989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:00.161730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4072a1-7ca8-420b-9092-04f3d05eb7c4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:00.170526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01576616821d494885b743bb4cf6f6a7', 'request_id': 'req-659096f0-63c7-413e-a6c2-c8c2cede154b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.182246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01576616821d494885b743bb4cf6f6a7', 'request_id': 'req-659096f0-63c7-413e-a6c2-c8c2cede154b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.191635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01576616821d494885b743bb4cf6f6a7', 'request_id': 'req-659096f0-63c7-413e-a6c2-c8c2cede154b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.252928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a848a-923f-4293-a8fd-b37d948ec77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:00.253624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a848a-923f-4293-a8fd-b37d948ec77a service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:00.253912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a848a-923f-4293-a8fd-b37d948ec77a service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:00.254217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a848a-923f-4293-a8fd-b37d948ec77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:00.283732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] The token cache is being invalidate because project 01576616821d494885b743bb4cf6f6a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:00.284585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.285611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.293026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 974/1923] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:01:59 2022] DELETE /identity/v3/projects/01576616821d494885b743bb4cf6f6a7 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:00.295258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:00.297095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:00.337283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a848a-923f-4293-a8fd-b37d948ec77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:00.338731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a848a-923f-4293-a8fd-b37d948ec77a service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:00.348393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 950/1924] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:00.381633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659096f0-63c7-413e-a6c2-c8c2cede154b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:00.390987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:00.392789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:00.393677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:00.395006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] PATH_INFO: `/v3/users/42bc55ed63af46aab5b5eddd0cc93464` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:00.397535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42bc55ed63af46aab5b5eddd0cc93464)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:00.401898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:00.460143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:00.614803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42bc55ed63af46aab5b5eddd0cc93464'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.631867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42bc55ed63af46aab5b5eddd0cc93464'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.639512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42bc55ed63af46aab5b5eddd0cc93464'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.640263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42bc55ed63af46aab5b5eddd0cc93464'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.753261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42bc55ed63af46aab5b5eddd0cc93464'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:00.767344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 975/1925] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:00 2022] DELETE /identity/v3/users/42bc55ed63af46aab5b5eddd0cc93464 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:00.872894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a848a-923f-4293-a8fd-b37d948ec77a service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:00.874844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2d4a848a-923f-4293-a8fd-b37d948ec77a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:00.888183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a848a-923f-4293-a8fd-b37d948ec77a service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:00.954948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:00.956440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:00.956974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:00.957476 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] PATH_INFO: `/v3/projects/fbf47a92239b40b19bd4e1cf35ebdce8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:00.960179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf47a92239b40b19bd4e1cf35ebdce8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:00.963958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:01.089363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf47a92239b40b19bd4e1cf35ebdce8', 'request_id': 'req-bcff5281-2c5f-4a37-a34b-32de84af700f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.099446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf47a92239b40b19bd4e1cf35ebdce8', 'request_id': 'req-bcff5281-2c5f-4a37-a34b-32de84af700f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.209941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf47a92239b40b19bd4e1cf35ebdce8', 'request_id': 'req-bcff5281-2c5f-4a37-a34b-32de84af700f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.225969 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] The token cache is being invalidate because project fbf47a92239b40b19bd4e1cf35ebdce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:01.226636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.227316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.229516 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 951/1926] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:00 2022] DELETE /identity/v3/projects/fbf47a92239b40b19bd4e1cf35ebdce8 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:01.233255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:01.235175 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:01.237902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:01.249075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287b65d-9e8a-42aa-83b7-9bc4e703f78e admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:01.291499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:01.321797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:01.321797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:01.321797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:01.321797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] PATH_INFO: `/v3/users/98ccd3cbb6dc4efbae634685b3227370` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:01.322157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ccd3cbb6dc4efbae634685b3227370)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:01.322157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:01.379616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:01.381839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:01.394501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff5281-2c5f-4a37-a34b-32de84af700f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:01.400253 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:01.401499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:01.401856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:01.402205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] PATH_INFO: `/v3/projects/533517fb139041fc8cdc568748aa83a2` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:01.403627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=533517fb139041fc8cdc568748aa83a2)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:01.405703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:01.566124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ccd3cbb6dc4efbae634685b3227370'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.574279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533517fb139041fc8cdc568748aa83a2', 'request_id': 'req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.582079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533517fb139041fc8cdc568748aa83a2', 'request_id': 'req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.587466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ccd3cbb6dc4efbae634685b3227370'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.594648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ccd3cbb6dc4efbae634685b3227370'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.596225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ccd3cbb6dc4efbae634685b3227370'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.653407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533517fb139041fc8cdc568748aa83a2', 'request_id': 'req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.664589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] The token cache is being invalidate because project 533517fb139041fc8cdc568748aa83a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:01.664909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.665550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.667140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 952/1927] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:01 2022] DELETE /identity/v3/projects/533517fb139041fc8cdc568748aa83a2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:01.670766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:01.670766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:01.741627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6bf36-be2e-4c54-8016-dc3d9a23ae25 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:01.747095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:01.748070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:01.748365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:01.748653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] PATH_INFO: `/v3/users/ca2d681ab7274bb9b7e403fb05ad01c5` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:01.749897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca2d681ab7274bb9b7e403fb05ad01c5)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:01.751815 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:01.768042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-19bb3472-e912-45a3-b1a6-92963c780bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ccd3cbb6dc4efbae634685b3227370'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.777286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 976/1928] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:01 2022] DELETE /identity/v3/users/98ccd3cbb6dc4efbae634685b3227370 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:01.780943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafff408-b8a7-4fa0-99d7-b1d81cabad49 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:01.781296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafff408-b8a7-4fa0-99d7-b1d81cabad49 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:01.787227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafff408-b8a7-4fa0-99d7-b1d81cabad49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:01.801469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-eafff408-b8a7-4fa0-99d7-b1d81cabad49 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:01.803407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-eafff408-b8a7-4fa0-99d7-b1d81cabad49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:01.874830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 977/1929] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:01.883272 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2a5b4-855a-4a18-9953-4a055c604db5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:01.883490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2a5b4-855a-4a18-9953-4a055c604db5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:01.883775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2a5b4-855a-4a18-9953-4a055c604db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:01.901975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a0e2a5b4-855a-4a18-9953-4a055c604db5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:01.903484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a0e2a5b4-855a-4a18-9953-4a055c604db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:01.911984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 978/1930] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:01.916263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2a5b4-855a-4a18-9953-4a055c604db5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:01.926571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2a5b4-855a-4a18-9953-4a055c604db5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:01.932269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979085e-b89a-46e4-9201-8a155a2ede4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:01.933126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979085e-b89a-46e4-9201-8a155a2ede4b admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:01.933498 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979085e-b89a-46e4-9201-8a155a2ede4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:01.933707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979085e-b89a-46e4-9201-8a155a2ede4b admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:01.934553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979085e-b89a-46e4-9201-8a155a2ede4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:01.934872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979085e-b89a-46e4-9201-8a155a2ede4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:01.936496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979085e-b89a-46e4-9201-8a155a2ede4b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:01.937762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca2d681ab7274bb9b7e403fb05ad01c5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.946485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca2d681ab7274bb9b7e403fb05ad01c5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.947252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca2d681ab7274bb9b7e403fb05ad01c5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:01.952691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 979/1931] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:02:01.956823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979085e-b89a-46e4-9201-8a155a2ede4b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:01.967020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979085e-b89a-46e4-9201-8a155a2ede4b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:01.971838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306e520-dac6-4d0d-af78-16d323f1f618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:01.974997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306e520-dac6-4d0d-af78-16d323f1f618 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:01.974997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306e520-dac6-4d0d-af78-16d323f1f618 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:01.974997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306e520-dac6-4d0d-af78-16d323f1f618 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:01.974997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306e520-dac6-4d0d-af78-16d323f1f618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:01.975768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306e520-dac6-4d0d-af78-16d323f1f618 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:02.052327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca2d681ab7274bb9b7e403fb05ad01c5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:02.067422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-be3fdb82-8bf6-4fe1-b03f-589c9c43105f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca2d681ab7274bb9b7e403fb05ad01c5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:02.076692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 953/1932] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:01 2022] DELETE /identity/v3/users/ca2d681ab7274bb9b7e403fb05ad01c5 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:02.079154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11b80e-06f1-40b5-bf04-77bdc0e26e8d None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.079452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11b80e-06f1-40b5-bf04-77bdc0e26e8d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.080884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11b80e-06f1-40b5-bf04-77bdc0e26e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.084336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 980/1933] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:01 2022] POST /identity/v3/projects => generated 397 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:02.088855 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306e520-dac6-4d0d-af78-16d323f1f618 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:02.091617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3306e520-dac6-4d0d-af78-16d323f1f618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:02.098472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-dc11b80e-06f1-40b5-bf04-77bdc0e26e8d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:02.100290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-dc11b80e-06f1-40b5-bf04-77bdc0e26e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:02.177656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 954/1934] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:02.188909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11b80e-06f1-40b5-bf04-77bdc0e26e8d None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:02.190887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-dc11b80e-06f1-40b5-bf04-77bdc0e26e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:02.196444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306e520-dac6-4d0d-af78-16d323f1f618 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:02.202609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d603e-98b9-4e2c-966d-aa198993b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:02.203307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d603e-98b9-4e2c-966d-aa198993b488 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.204167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d603e-98b9-4e2c-966d-aa198993b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.204476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d603e-98b9-4e2c-966d-aa198993b488 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.205369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d603e-98b9-4e2c-966d-aa198993b488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:02.208381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d603e-98b9-4e2c-966d-aa198993b488 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:02.251875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11b80e-06f1-40b5-bf04-77bdc0e26e8d None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:02.256319 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59aa27-bda3-4e2d-9e68-3cd30c7d1898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:02.257042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59aa27-bda3-4e2d-9e68-3cd30c7d1898 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.257332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59aa27-bda3-4e2d-9e68-3cd30c7d1898 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.257608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59aa27-bda3-4e2d-9e68-3cd30c7d1898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.267292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59aa27-bda3-4e2d-9e68-3cd30c7d1898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:02.268763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59aa27-bda3-4e2d-9e68-3cd30c7d1898 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:02.278034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 955/1935] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:02.288960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 981/1936] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:02 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:02.293039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59aa27-bda3-4e2d-9e68-3cd30c7d1898 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:02.294419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d603e-98b9-4e2c-966d-aa198993b488 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:02.296431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4f6d603e-98b9-4e2c-966d-aa198993b488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:02.303881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59aa27-bda3-4e2d-9e68-3cd30c7d1898 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:02.309359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b8eb4-83fd-4eae-b889-1d13408aeba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:02.310030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b8eb4-83fd-4eae-b889-1d13408aeba2 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.310162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d603e-98b9-4e2c-966d-aa198993b488 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:02.310316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b8eb4-83fd-4eae-b889-1d13408aeba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.310670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b8eb4-83fd-4eae-b889-1d13408aeba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.311401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b8eb4-83fd-4eae-b889-1d13408aeba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:02.313094 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b8eb4-83fd-4eae-b889-1d13408aeba2 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:02.318987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:02.318987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.318987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.318987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] PATH_INFO: `/v3/projects/209bfbb387514f499868a78461fcd568` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.319364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] RBAC: Authorizing `identity:delete_project(project_id=209bfbb387514f499868a78461fcd568)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:02.320650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:02.334365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 956/1937] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:02.340168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b8eb4-83fd-4eae-b889-1d13408aeba2 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:02.353869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b8eb4-83fd-4eae-b889-1d13408aeba2 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:02.367722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3d247-5435-48c9-994c-9e1031dc90f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:02.372616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3d247-5435-48c9-994c-9e1031dc90f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.372616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3d247-5435-48c9-994c-9e1031dc90f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.372616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3d247-5435-48c9-994c-9e1031dc90f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.372616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3d247-5435-48c9-994c-9e1031dc90f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:02.372616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3d247-5435-48c9-994c-9e1031dc90f3 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:02.400495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 957/1938] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:02.406353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3d247-5435-48c9-994c-9e1031dc90f3 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:02.419735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3d247-5435-48c9-994c-9e1031dc90f3 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:02.434897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90060259-f4fa-45d9-aacc-5bc70347a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:02.435341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90060259-f4fa-45d9-aacc-5bc70347a0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.435633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90060259-f4fa-45d9-aacc-5bc70347a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.435913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90060259-f4fa-45d9-aacc-5bc70347a0b9 admin admin] PATH_INFO: `/v3/projects/58e7d07a9c8a4543942ef788fef8fba1/users/c359517bc5904f85919773f3d6816fe5/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.451741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90060259-f4fa-45d9-aacc-5bc70347a0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58e7d07a9c8a4543942ef788fef8fba1, user_id=c359517bc5904f85919773f3d6816fe5, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:02.454451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90060259-f4fa-45d9-aacc-5bc70347a0b9 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:02.560231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 958/1939] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:02 2022] PUT /identity/v3/projects/58e7d07a9c8a4543942ef788fef8fba1/users/c359517bc5904f85919773f3d6816fe5/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:02.578656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62b154-9306-47cf-8636-5174107a780f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.579655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62b154-9306-47cf-8636-5174107a780f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.579944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62b154-9306-47cf-8636-5174107a780f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.586729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209bfbb387514f499868a78461fcd568', 'request_id': 'req-b8740b0a-0f9b-4a06-af41-9eed1377741a'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:02.592920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209bfbb387514f499868a78461fcd568', 'request_id': 'req-b8740b0a-0f9b-4a06-af41-9eed1377741a'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:02.601537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209bfbb387514f499868a78461fcd568', 'request_id': 'req-b8740b0a-0f9b-4a06-af41-9eed1377741a'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:02.622301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ae62b154-9306-47cf-8636-5174107a780f None None] MFA Rules not processed for user `c359517bc5904f85919773f3d6816fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:02.623567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ae62b154-9306-47cf-8636-5174107a780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:02.695406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] The token cache is being invalidate because project 209bfbb387514f499868a78461fcd568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:02.696014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:02.696930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b8740b0a-0f9b-4a06-af41-9eed1377741a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:02.698990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 982/1940] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:02 2022] DELETE /identity/v3/projects/209bfbb387514f499868a78461fcd568 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:02.704180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6934f-dcaf-4b5e-b646-42d0e9db72e7 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.705047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6934f-dcaf-4b5e-b646-42d0e9db72e7 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.705660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6934f-dcaf-4b5e-b646-42d0e9db72e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.713358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 959/1941] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:02.718909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:02.720824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62b154-9306-47cf-8636-5174107a780f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:02.745800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-8ea6934f-dcaf-4b5e-b646-42d0e9db72e7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:02.747261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8ea6934f-dcaf-4b5e-b646-42d0e9db72e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:02.804223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:02.821645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62b154-9306-47cf-8636-5174107a780f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:02.831582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bbc98-eab4-47da-9aab-b87f52648402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:02.832501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bbc98-eab4-47da-9aab-b87f52648402 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.832862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bbc98-eab4-47da-9aab-b87f52648402 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.833147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bbc98-eab4-47da-9aab-b87f52648402 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.836927 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bbc98-eab4-47da-9aab-b87f52648402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:02.840153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bbc98-eab4-47da-9aab-b87f52648402 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:02.866245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 983/1942] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:02.875351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a59e6-b50d-4604-bf51-c971028ce69f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.876536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a59e6-b50d-4604-bf51-c971028ce69f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.876536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a59e6-b50d-4604-bf51-c971028ce69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.898807 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-b72a59e6-b50d-4604-bf51-c971028ce69f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:02.900121 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b72a59e6-b50d-4604-bf51-c971028ce69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:02.915034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 984/1943] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:02.917411 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a59e6-b50d-4604-bf51-c971028ce69f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:02.944448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a59e6-b50d-4604-bf51-c971028ce69f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:02.950309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51449037-00f4-42d2-a909-b6e78c637068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:02.951102 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51449037-00f4-42d2-a909-b6e78c637068 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:02.951462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51449037-00f4-42d2-a909-b6e78c637068 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:02.951981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51449037-00f4-42d2-a909-b6e78c637068 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:02.952938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51449037-00f4-42d2-a909-b6e78c637068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:02.953312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51449037-00f4-42d2-a909-b6e78c637068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:02.955062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51449037-00f4-42d2-a909-b6e78c637068 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:02.972717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 985/1944] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:02:02.974551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51449037-00f4-42d2-a909-b6e78c637068 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:02.986415 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51449037-00f4-42d2-a909-b6e78c637068 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.003968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec94f6-c01f-4e88-a70f-4f116ee74251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.005097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec94f6-c01f-4e88-a70f-4f116ee74251 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.005378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec94f6-c01f-4e88-a70f-4f116ee74251 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.005659 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec94f6-c01f-4e88-a70f-4f116ee74251 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.007039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec94f6-c01f-4e88-a70f-4f116ee74251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.010294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 960/1945] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:02 2022] POST /identity/v3/projects => generated 401 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:03.013650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec94f6-c01f-4e88-a70f-4f116ee74251 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.014548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bbc98-eab4-47da-9aab-b87f52648402 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:03.019435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-593bbc98-eab4-47da-9aab-b87f52648402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:03.132279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bbc98-eab4-47da-9aab-b87f52648402 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.137109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dd5fc-a1f1-4b62-b110-1fc51858727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.140086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dd5fc-a1f1-4b62-b110-1fc51858727b admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.140086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dd5fc-a1f1-4b62-b110-1fc51858727b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.140086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dd5fc-a1f1-4b62-b110-1fc51858727b admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.140086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dd5fc-a1f1-4b62-b110-1fc51858727b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.141500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dd5fc-a1f1-4b62-b110-1fc51858727b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.243655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 986/1946] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:02 2022] POST /identity/v3/projects => generated 379 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:03.244907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec94f6-c01f-4e88-a70f-4f116ee74251 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:03.283145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 961/1947] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:03 2022] POST /identity/v3/users => generated 307 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:03.285325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dd5fc-a1f1-4b62-b110-1fc51858727b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:03.289918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-eb5dd5fc-a1f1-4b62-b110-1fc51858727b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:03.321698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dd5fc-a1f1-4b62-b110-1fc51858727b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.378665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec94f6-c01f-4e88-a70f-4f116ee74251 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.403174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.413384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.413776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.414151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] PATH_INFO: `/v3/projects/99136a2dbbfc49a2bccce9d17eb5d6a6` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.415639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99136a2dbbfc49a2bccce9d17eb5d6a6)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.417923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.423128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec7632-8d96-4ff8-b50e-c1e8728c63bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.431191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec7632-8d96-4ff8-b50e-c1e8728c63bb admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.431636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec7632-8d96-4ff8-b50e-c1e8728c63bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.432072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec7632-8d96-4ff8-b50e-c1e8728c63bb admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.435913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec7632-8d96-4ff8-b50e-c1e8728c63bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.441551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec7632-8d96-4ff8-b50e-c1e8728c63bb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.548333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 987/1948] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:03 2022] POST /identity/v3/users => generated 290 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:03.549796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec7632-8d96-4ff8-b50e-c1e8728c63bb admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:03.561851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99136a2dbbfc49a2bccce9d17eb5d6a6', 'request_id': 'req-737048f6-412f-4530-be3c-7ce55eddc001'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:03.562408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec7632-8d96-4ff8-b50e-c1e8728c63bb admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.569920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99136a2dbbfc49a2bccce9d17eb5d6a6', 'request_id': 'req-737048f6-412f-4530-be3c-7ce55eddc001'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:03.634547 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f360b5-8abd-416e-a3c6-08cf5bf88e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.636962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f360b5-8abd-416e-a3c6-08cf5bf88e5d admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.637495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f360b5-8abd-416e-a3c6-08cf5bf88e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.637989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f360b5-8abd-416e-a3c6-08cf5bf88e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.639454 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f360b5-8abd-416e-a3c6-08cf5bf88e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.641589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f360b5-8abd-416e-a3c6-08cf5bf88e5d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.656346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 988/1949] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:03.656346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f360b5-8abd-416e-a3c6-08cf5bf88e5d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:03.667385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f360b5-8abd-416e-a3c6-08cf5bf88e5d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.672922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914ef0a-314a-46dc-ac16-ac6b93e4016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.673803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ef0a-314a-46dc-ac16-ac6b93e4016c admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.674100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ef0a-314a-46dc-ac16-ac6b93e4016c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.674383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ef0a-314a-46dc-ac16-ac6b93e4016c admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.674819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99136a2dbbfc49a2bccce9d17eb5d6a6', 'request_id': 'req-737048f6-412f-4530-be3c-7ce55eddc001'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:03.675120 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914ef0a-314a-46dc-ac16-ac6b93e4016c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.676844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914ef0a-314a-46dc-ac16-ac6b93e4016c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.687899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] The token cache is being invalidate because project 99136a2dbbfc49a2bccce9d17eb5d6a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:03.688198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:03.688981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:03.689886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 989/1950] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:03.690556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914ef0a-314a-46dc-ac16-ac6b93e4016c admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:03.690691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 962/1951] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:03 2022] DELETE /identity/v3/projects/99136a2dbbfc49a2bccce9d17eb5d6a6 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:03.693605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:03.695058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:03.695174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1914ef0a-314a-46dc-ac16-ac6b93e4016c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:03.709813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914ef0a-314a-46dc-ac16-ac6b93e4016c admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.716835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737048f6-412f-4530-be3c-7ce55eddc001 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.719361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df050a-963c-40a2-b564-6c04ff5ea8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.720249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df050a-963c-40a2-b564-6c04ff5ea8ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.720534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df050a-963c-40a2-b564-6c04ff5ea8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.720859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df050a-963c-40a2-b564-6c04ff5ea8ee admin admin] PATH_INFO: `/v3/projects/7df1cbc1c4d4475a89afee39870ccc22/users/50b135d868884b87a98ca54911596936/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.734820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e0954-aef2-4222-983c-907da84001ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.735689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e0954-aef2-4222-983c-907da84001ca admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.736784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e0954-aef2-4222-983c-907da84001ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.736784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e0954-aef2-4222-983c-907da84001ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.737756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e0954-aef2-4222-983c-907da84001ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.739475 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df050a-963c-40a2-b564-6c04ff5ea8ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df1cbc1c4d4475a89afee39870ccc22, user_id=50b135d868884b87a98ca54911596936, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.742148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e0954-aef2-4222-983c-907da84001ca admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.742912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df050a-963c-40a2-b564-6c04ff5ea8ee admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.760728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 963/1952] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:03.760728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e0954-aef2-4222-983c-907da84001ca admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:03.761526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f19e0954-aef2-4222-983c-907da84001ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:03.775902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e0954-aef2-4222-983c-907da84001ca admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.782850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.784322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.784322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.784718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] PATH_INFO: `/v3/users/412b0fa191d24b2f82b7299c4c289c66` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.786035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=412b0fa191d24b2f82b7299c4c289c66)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.788105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.811159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 990/1953] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:03 2022] PUT /identity/v3/projects/7df1cbc1c4d4475a89afee39870ccc22/users/50b135d868884b87a98ca54911596936/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:03.811159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df050a-963c-40a2-b564-6c04ff5ea8ee admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:03.817600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:03.874293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df050a-963c-40a2-b564-6c04ff5ea8ee admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:03.878959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991e3d8-30d6-4262-9f8d-395825b60971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:03.879861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991e3d8-30d6-4262-9f8d-395825b60971 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:03.880142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991e3d8-30d6-4262-9f8d-395825b60971 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:03.880420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991e3d8-30d6-4262-9f8d-395825b60971 admin admin] PATH_INFO: `/v3/projects/db721455223c4f11addbdd66b5ed41a1/users/a281cfbbaab44541b8e1d4fef6b86acf/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:03.896141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991e3d8-30d6-4262-9f8d-395825b60971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db721455223c4f11addbdd66b5ed41a1, user_id=a281cfbbaab44541b8e1d4fef6b86acf, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:03.899601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991e3d8-30d6-4262-9f8d-395825b60971 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:03.936282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '412b0fa191d24b2f82b7299c4c289c66'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:03.944077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '412b0fa191d24b2f82b7299c4c289c66'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:03.944857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '412b0fa191d24b2f82b7299c4c289c66'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:03.985868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 991/1954] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:03 2022] PUT /identity/v3/projects/db721455223c4f11addbdd66b5ed41a1/users/a281cfbbaab44541b8e1d4fef6b86acf/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:03.986802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991e3d8-30d6-4262-9f8d-395825b60971 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:04.025317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '412b0fa191d24b2f82b7299c4c289c66'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:04.042170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7cdee517-b86b-4adc-8933-ce0480301629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '412b0fa191d24b2f82b7299c4c289c66'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:04.049719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 964/1955] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:03 2022] DELETE /identity/v3/users/412b0fa191d24b2f82b7299c4c289c66 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:04.053851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06409b33-028c-4af3-8aab-0ac751804805 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.053851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06409b33-028c-4af3-8aab-0ac751804805 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.053851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06409b33-028c-4af3-8aab-0ac751804805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.056685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991e3d8-30d6-4262-9f8d-395825b60971 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:04.086571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-06409b33-028c-4af3-8aab-0ac751804805 None None] MFA Rules not processed for user `a281cfbbaab44541b8e1d4fef6b86acf`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:04.087851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-06409b33-028c-4af3-8aab-0ac751804805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:04.142491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8952842-d96f-4cda-a673-b5eda4ff4845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:04.143306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8952842-d96f-4cda-a673-b5eda4ff4845 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.144913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:04.145714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8952842-d96f-4cda-a673-b5eda4ff4845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.146087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8952842-d96f-4cda-a673-b5eda4ff4845 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.147133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8952842-d96f-4cda-a673-b5eda4ff4845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:04.148939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8952842-d96f-4cda-a673-b5eda4ff4845 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:04.161941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 992/1956] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:04.170420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8952842-d96f-4cda-a673-b5eda4ff4845 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:04.175800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e8952842-d96f-4cda-a673-b5eda4ff4845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:04.191325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8952842-d96f-4cda-a673-b5eda4ff4845 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:04.191697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 965/1957] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:04.196879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06409b33-028c-4af3-8aab-0ac751804805 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:04.197682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e962ccf-ae1a-4e3b-83d0-28982d03093a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:04.198472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-06409b33-028c-4af3-8aab-0ac751804805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:04.198647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e962ccf-ae1a-4e3b-83d0-28982d03093a admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.198901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e962ccf-ae1a-4e3b-83d0-28982d03093a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.199179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e962ccf-ae1a-4e3b-83d0-28982d03093a admin admin] PATH_INFO: `/v3/projects/7df1cbc1c4d4475a89afee39870ccc22/users/50b135d868884b87a98ca54911596936/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.201399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e962ccf-ae1a-4e3b-83d0-28982d03093a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df1cbc1c4d4475a89afee39870ccc22, user_id=50b135d868884b87a98ca54911596936, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:04.204710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e962ccf-ae1a-4e3b-83d0-28982d03093a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:04.216115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06409b33-028c-4af3-8aab-0ac751804805 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:04.224869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 993/1958] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:04 2022] PUT /identity/v3/projects/7df1cbc1c4d4475a89afee39870ccc22/users/50b135d868884b87a98ca54911596936/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:04.230905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3c927-9b47-4ee3-9f51-a1826eff279b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.232027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3c927-9b47-4ee3-9f51-a1826eff279b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.239498 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3c927-9b47-4ee3-9f51-a1826eff279b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.281207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-62d3c927-9b47-4ee3-9f51-a1826eff279b None None] MFA Rules not processed for user `50b135d868884b87a98ca54911596936`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:04.283443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-62d3c927-9b47-4ee3-9f51-a1826eff279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:04.304162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aac87f-ba4d-46f5-ac24-0d8426ed3124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:04.305774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aac87f-ba4d-46f5-ac24-0d8426ed3124 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.306086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aac87f-ba4d-46f5-ac24-0d8426ed3124 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.306512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aac87f-ba4d-46f5-ac24-0d8426ed3124 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.307845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aac87f-ba4d-46f5-ac24-0d8426ed3124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:04.312983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aac87f-ba4d-46f5-ac24-0d8426ed3124 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:04.374357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 994/1959] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:04 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:04.388292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3c927-9b47-4ee3-9f51-a1826eff279b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:04.390289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-62d3c927-9b47-4ee3-9f51-a1826eff279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:04.421470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 966/1960] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:04 2022] POST /identity/v3/projects => generated 379 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:04.431128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aac87f-ba4d-46f5-ac24-0d8426ed3124 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:04.466127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3c927-9b47-4ee3-9f51-a1826eff279b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:04.541416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aac87f-ba4d-46f5-ac24-0d8426ed3124 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:04.557893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7412e-b1c9-4f34-ab50-870bb8271bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:04.559025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7412e-b1c9-4f34-ab50-870bb8271bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.559025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7412e-b1c9-4f34-ab50-870bb8271bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.559182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7412e-b1c9-4f34-ab50-870bb8271bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.560073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7412e-b1c9-4f34-ab50-870bb8271bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:04.563606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7412e-b1c9-4f34-ab50-870bb8271bb0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:04.575767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681f49f-cfb9-44f2-8aea-f42e08a610f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:04.617774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681f49f-cfb9-44f2-8aea-f42e08a610f2 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.617774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681f49f-cfb9-44f2-8aea-f42e08a610f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.617774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681f49f-cfb9-44f2-8aea-f42e08a610f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.690613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681f49f-cfb9-44f2-8aea-f42e08a610f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:04.692423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681f49f-cfb9-44f2-8aea-f42e08a610f2 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:04.703551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 995/1961] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:04 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:04.805537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681f49f-cfb9-44f2-8aea-f42e08a610f2 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:04.807874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 967/1962] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:04 2022] POST /identity/v3/users => generated 296 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:04.818753 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681f49f-cfb9-44f2-8aea-f42e08a610f2 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:04.837380 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7412e-b1c9-4f34-ab50-870bb8271bb0 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:04.840245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c8d7412e-b1c9-4f34-ab50-870bb8271bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:04.861736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa7b2-6c26-4dd8-bde6-7d5759f1abf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:04.861736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa7b2-6c26-4dd8-bde6-7d5759f1abf4 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.861736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa7b2-6c26-4dd8-bde6-7d5759f1abf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.861736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa7b2-6c26-4dd8-bde6-7d5759f1abf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.870460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afa7b2-6c26-4dd8-bde6-7d5759f1abf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:04.875907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afa7b2-6c26-4dd8-bde6-7d5759f1abf4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:04.882039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7412e-b1c9-4f34-ab50-870bb8271bb0 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:04.891001 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:04.892051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.892413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.892803 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] PATH_INFO: `/v3/projects/6c19c512fb284410ab35ae2f4e3cc0b4` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.897798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c19c512fb284410ab35ae2f4e3cc0b4)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:04.897975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 996/1963] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:04.899051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa7b2-6c26-4dd8-bde6-7d5759f1abf4 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:04.900192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:04.928688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa7b2-6c26-4dd8-bde6-7d5759f1abf4 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:04.941289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a8410-18a5-444f-a6cb-7a1b3b20f7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:04.942378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8410-18a5-444f-a6cb-7a1b3b20f7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:04.942702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8410-18a5-444f-a6cb-7a1b3b20f7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:04.943180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8410-18a5-444f-a6cb-7a1b3b20f7e1 admin admin] PATH_INFO: `/v3/projects/f890f20b9e2c4f32a9ff87f8e1848cba/users/98e93f83bfcb432481e412f4e239eedd/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:04.987869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a8410-18a5-444f-a6cb-7a1b3b20f7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f890f20b9e2c4f32a9ff87f8e1848cba, user_id=98e93f83bfcb432481e412f4e239eedd, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:04.991087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a8410-18a5-444f-a6cb-7a1b3b20f7e1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:05.092004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 997/1964] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:04 2022] PUT /identity/v3/projects/f890f20b9e2c4f32a9ff87f8e1848cba/users/98e93f83bfcb432481e412f4e239eedd/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:05.092977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a8410-18a5-444f-a6cb-7a1b3b20f7e1 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:05.120495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c19c512fb284410ab35ae2f4e3cc0b4', 'request_id': 'req-fd10802e-0257-479e-a4c5-599d46ec37c1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:05.135093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c19c512fb284410ab35ae2f4e3cc0b4', 'request_id': 'req-fd10802e-0257-479e-a4c5-599d46ec37c1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:05.156655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:05.207840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c19c512fb284410ab35ae2f4e3cc0b4', 'request_id': 'req-fd10802e-0257-479e-a4c5-599d46ec37c1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:05.208794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a8410-18a5-444f-a6cb-7a1b3b20f7e1 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:05.221461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] The token cache is being invalidate because project 6c19c512fb284410ab35ae2f4e3cc0b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:05.221845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:05.222993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:05.227902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 968/1965] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:04 2022] DELETE /identity/v3/projects/6c19c512fb284410ab35ae2f4e3cc0b4 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:05.227902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:05.229886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:05.284271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cffd3-4d8a-4752-823d-44ec82014974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:05.285264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cffd3-4d8a-4752-823d-44ec82014974 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:05.285689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cffd3-4d8a-4752-823d-44ec82014974 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:05.286221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cffd3-4d8a-4752-823d-44ec82014974 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:05.287015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cffd3-4d8a-4752-823d-44ec82014974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:05.288689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cffd3-4d8a-4752-823d-44ec82014974 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:05.301737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10802e-0257-479e-a4c5-599d46ec37c1 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:05.305946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d218e6c-caa7-4735-8eb5-7ffe4673546a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:05.308793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d218e6c-caa7-4735-8eb5-7ffe4673546a service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:05.308793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d218e6c-caa7-4735-8eb5-7ffe4673546a service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:05.308793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d218e6c-caa7-4735-8eb5-7ffe4673546a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:05.309077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1d218e6c-caa7-4735-8eb5-7ffe4673546a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:05.311072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 998/1966] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:05.313054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540ce34-0bf8-457f-8be2-2b014bdd6eca None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:05.313614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540ce34-0bf8-457f-8be2-2b014bdd6eca None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:05.330559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540ce34-0bf8-457f-8be2-2b014bdd6eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:05.366988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1540ce34-0bf8-457f-8be2-2b014bdd6eca None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:05.369160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1540ce34-0bf8-457f-8be2-2b014bdd6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:05.392386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 999/1967] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:05.392386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540ce34-0bf8-457f-8be2-2b014bdd6eca None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:05.392386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1540ce34-0bf8-457f-8be2-2b014bdd6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:05.409970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540ce34-0bf8-457f-8be2-2b014bdd6eca None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:05.416503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15dcc1f-371e-4901-b0d7-db6df27e6d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:05.419105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dcc1f-371e-4901-b0d7-db6df27e6d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:05.420228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dcc1f-371e-4901-b0d7-db6df27e6d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:05.420621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dcc1f-371e-4901-b0d7-db6df27e6d43 admin admin] PATH_INFO: `/v3/projects/f890f20b9e2c4f32a9ff87f8e1848cba/users/98e93f83bfcb432481e412f4e239eedd/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:05.429514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15dcc1f-371e-4901-b0d7-db6df27e6d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f890f20b9e2c4f32a9ff87f8e1848cba, user_id=98e93f83bfcb432481e412f4e239eedd, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:05.433581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15dcc1f-371e-4901-b0d7-db6df27e6d43 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:05.435268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d218e6c-caa7-4735-8eb5-7ffe4673546a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:05.436951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d218e6c-caa7-4735-8eb5-7ffe4673546a service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:05.444204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:05.448803 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 969/1968] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:05.449980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbff44-5adb-4be3-b371-d1457cfd3dd9 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:05.450290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbff44-5adb-4be3-b371-d1457cfd3dd9 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:05.450613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbff44-5adb-4be3-b371-d1457cfd3dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:05.471698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1000/1969] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:05 2022] PUT /identity/v3/projects/f890f20b9e2c4f32a9ff87f8e1848cba/users/98e93f83bfcb432481e412f4e239eedd/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:05.473120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-facbff44-5adb-4be3-b371-d1457cfd3dd9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:05.474376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-facbff44-5adb-4be3-b371-d1457cfd3dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:05.483620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2322a7-905a-49e0-81d8-6fe1c9cf6290 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:05.484225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2322a7-905a-49e0-81d8-6fe1c9cf6290 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:05.488702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2322a7-905a-49e0-81d8-6fe1c9cf6290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:05.553628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:05.564213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-db2322a7-905a-49e0-81d8-6fe1c9cf6290 None None] MFA Rules not processed for user `98e93f83bfcb432481e412f4e239eedd`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:05.565678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-db2322a7-905a-49e0-81d8-6fe1c9cf6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:05.622549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:05.638618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 970/1970] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:05.645125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbff44-5adb-4be3-b371-d1457cfd3dd9 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:05.663269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbff44-5adb-4be3-b371-d1457cfd3dd9 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:05.689223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80642c-9bf4-48f0-af8d-050cc9b82740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:05.691517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80642c-9bf4-48f0-af8d-050cc9b82740 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:05.692451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80642c-9bf4-48f0-af8d-050cc9b82740 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:05.693043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80642c-9bf4-48f0-af8d-050cc9b82740 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:05.697950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80642c-9bf4-48f0-af8d-050cc9b82740 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:05.699459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80642c-9bf4-48f0-af8d-050cc9b82740 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:05.704134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80642c-9bf4-48f0-af8d-050cc9b82740 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:05.753093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 971/1971] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:02:05.757884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80642c-9bf4-48f0-af8d-050cc9b82740 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:05.770927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1001/1972] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:05.782681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2322a7-905a-49e0-81d8-6fe1c9cf6290 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:05.787339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80642c-9bf4-48f0-af8d-050cc9b82740 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:05.789094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-db2322a7-905a-49e0-81d8-6fe1c9cf6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:05.797766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b109c5-bdbb-46f0-bf5d-b926d9471006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:05.798590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b109c5-bdbb-46f0-bf5d-b926d9471006 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:05.799249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b109c5-bdbb-46f0-bf5d-b926d9471006 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:05.799603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b109c5-bdbb-46f0-bf5d-b926d9471006 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:05.800452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b109c5-bdbb-46f0-bf5d-b926d9471006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:05.803716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b109c5-bdbb-46f0-bf5d-b926d9471006 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:05.838399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:05.902615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2322a7-905a-49e0-81d8-6fe1c9cf6290 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:05.906756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71932a0-1598-4143-95c0-665a5a907d77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:05.907428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71932a0-1598-4143-95c0-665a5a907d77 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:05.908166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71932a0-1598-4143-95c0-665a5a907d77 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:05.908166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71932a0-1598-4143-95c0-665a5a907d77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:05.918600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71932a0-1598-4143-95c0-665a5a907d77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:05.920311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71932a0-1598-4143-95c0-665a5a907d77 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:05.928562 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:05.934184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1002/1973] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:05.940682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 972/1974] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:05 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:05.945392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71932a0-1598-4143-95c0-665a5a907d77 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:05.956865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71932a0-1598-4143-95c0-665a5a907d77 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:06.022343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553de8a-3edc-48a5-955d-fb8c604e7989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:06.024388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553de8a-3edc-48a5-955d-fb8c604e7989 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:06.024388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553de8a-3edc-48a5-955d-fb8c604e7989 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:06.024388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553de8a-3edc-48a5-955d-fb8c604e7989 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:06.024761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553de8a-3edc-48a5-955d-fb8c604e7989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:06.027157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553de8a-3edc-48a5-955d-fb8c604e7989 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:06.112066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1003/1975] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:05 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:06.112416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b109c5-bdbb-46f0-bf5d-b926d9471006 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:06.132427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b109c5-bdbb-46f0-bf5d-b926d9471006 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:06.138248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b09d86-1cd8-4586-a881-e98ab7abd78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:06.139035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09d86-1cd8-4586-a881-e98ab7abd78c admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:06.139326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09d86-1cd8-4586-a881-e98ab7abd78c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:06.139606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09d86-1cd8-4586-a881-e98ab7abd78c admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:06.140404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b09d86-1cd8-4586-a881-e98ab7abd78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:06.144932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b09d86-1cd8-4586-a881-e98ab7abd78c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:06.161521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 973/1976] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:02:06.167316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553de8a-3edc-48a5-955d-fb8c604e7989 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:06.180856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553de8a-3edc-48a5-955d-fb8c604e7989 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:06.185980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e2250-c05d-4a23-b217-0c1e8f678007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:06.186755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e2250-c05d-4a23-b217-0c1e8f678007 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:06.187112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e2250-c05d-4a23-b217-0c1e8f678007 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:06.187461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e2250-c05d-4a23-b217-0c1e8f678007 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:06.188384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e2250-c05d-4a23-b217-0c1e8f678007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:06.190109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e2250-c05d-4a23-b217-0c1e8f678007 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:06.201052 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1004/1977] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:06.205254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b09d86-1cd8-4586-a881-e98ab7abd78c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:06.215949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b09d86-1cd8-4586-a881-e98ab7abd78c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:06.221077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec6c5-6944-40a2-9f18-11dde9388f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:06.222126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec6c5-6944-40a2-9f18-11dde9388f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:06.222262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec6c5-6944-40a2-9f18-11dde9388f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:06.222542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec6c5-6944-40a2-9f18-11dde9388f8c admin admin] PATH_INFO: `/v3/projects/a89d390893f3469eb01534a788e9e1eb/users/850e54d515fd4233bbc46fc72ab41812/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:06.240277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ec6c5-6944-40a2-9f18-11dde9388f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89d390893f3469eb01534a788e9e1eb, user_id=850e54d515fd4233bbc46fc72ab41812, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:06.243126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ec6c5-6944-40a2-9f18-11dde9388f8c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:06.270493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:06.357217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 974/1978] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:06 2022] PUT /identity/v3/projects/a89d390893f3469eb01534a788e9e1eb/users/850e54d515fd4233bbc46fc72ab41812/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:06.367844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a34dd-c0b7-4a2a-bc8b-c24cab7d1b46 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:06.368134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a34dd-c0b7-4a2a-bc8b-c24cab7d1b46 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:06.368415 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a34dd-c0b7-4a2a-bc8b-c24cab7d1b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:06.408833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-9d7a34dd-c0b7-4a2a-bc8b-c24cab7d1b46 None None] MFA Rules not processed for user `850e54d515fd4233bbc46fc72ab41812`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:06.411903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9d7a34dd-c0b7-4a2a-bc8b-c24cab7d1b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:06.477145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:06.483255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1005/1979] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:06.487866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec6c5-6944-40a2-9f18-11dde9388f8c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:06.566584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec6c5-6944-40a2-9f18-11dde9388f8c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:06.577480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920368f-3496-473a-9916-524ff19b49a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:06.578355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920368f-3496-473a-9916-524ff19b49a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:06.578748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920368f-3496-473a-9916-524ff19b49a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:06.579015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920368f-3496-473a-9916-524ff19b49a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:06.581163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920368f-3496-473a-9916-524ff19b49a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:06.581945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920368f-3496-473a-9916-524ff19b49a3 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:06.765993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 975/1980] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:06 2022] POST /identity/v3/projects => generated 385 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:06.770059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a34dd-c0b7-4a2a-bc8b-c24cab7d1b46 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:06.785481 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a34dd-c0b7-4a2a-bc8b-c24cab7d1b46 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:06.854382 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e0bc8-a54d-4a90-b4ce-d4661222d478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:06.855047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e0bc8-a54d-4a90-b4ce-d4661222d478 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:06.855333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e0bc8-a54d-4a90-b4ce-d4661222d478 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:06.855615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e0bc8-a54d-4a90-b4ce-d4661222d478 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:06.856308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e0bc8-a54d-4a90-b4ce-d4661222d478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:06.858271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e0bc8-a54d-4a90-b4ce-d4661222d478 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:06.887615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920368f-3496-473a-9916-524ff19b49a3 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:06.899474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920368f-3496-473a-9916-524ff19b49a3 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:06.904376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:06.905550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:06.906465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:06.906465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] PATH_INFO: `/v3/users/a281cfbbaab44541b8e1d4fef6b86acf` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:06.907472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a281cfbbaab44541b8e1d4fef6b86acf)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:06.910133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:06.974709 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:06.985744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1006/1981] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:06 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:06.986935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e0bc8-a54d-4a90-b4ce-d4661222d478 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:06.998482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e0bc8-a54d-4a90-b4ce-d4661222d478 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:07.003620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b979985-4703-42fa-8eeb-a92385e672cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:07.004504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b979985-4703-42fa-8eeb-a92385e672cc admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.005021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b979985-4703-42fa-8eeb-a92385e672cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.008349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b979985-4703-42fa-8eeb-a92385e672cc admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.008349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b979985-4703-42fa-8eeb-a92385e672cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:07.008605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b979985-4703-42fa-8eeb-a92385e672cc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:07.020719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1007/1982] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:07.023868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b979985-4703-42fa-8eeb-a92385e672cc admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:07.033829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b979985-4703-42fa-8eeb-a92385e672cc admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:07.039559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab214d8-b20e-4fe2-8c86-e5fdf3902b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:07.040314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab214d8-b20e-4fe2-8c86-e5fdf3902b46 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.040782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab214d8-b20e-4fe2-8c86-e5fdf3902b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.041064 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab214d8-b20e-4fe2-8c86-e5fdf3902b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.041895 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab214d8-b20e-4fe2-8c86-e5fdf3902b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:07.043551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab214d8-b20e-4fe2-8c86-e5fdf3902b46 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:07.055513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1008/1983] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:02:07.069591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab214d8-b20e-4fe2-8c86-e5fdf3902b46 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:07.079372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab214d8-b20e-4fe2-8c86-e5fdf3902b46 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:07.086823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363bdb1-a623-4752-9a7d-99aa9974c0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:07.088909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363bdb1-a623-4752-9a7d-99aa9974c0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.088909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363bdb1-a623-4752-9a7d-99aa9974c0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.088909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363bdb1-a623-4752-9a7d-99aa9974c0e8 admin admin] PATH_INFO: `/v3/projects/111b344df3f84d0591531cc33acefdfd/users/5894cc21fe2b4c9eb4f6695f0e772d27/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.103961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363bdb1-a623-4752-9a7d-99aa9974c0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111b344df3f84d0591531cc33acefdfd, user_id=5894cc21fe2b4c9eb4f6695f0e772d27, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:07.106739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363bdb1-a623-4752-9a7d-99aa9974c0e8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:07.180945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a281cfbbaab44541b8e1d4fef6b86acf'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:07.187974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a281cfbbaab44541b8e1d4fef6b86acf'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:07.188908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a281cfbbaab44541b8e1d4fef6b86acf'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:07.253431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:07.260098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1009/1984] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:07 2022] PUT /identity/v3/projects/111b344df3f84d0591531cc33acefdfd/users/5894cc21fe2b4c9eb4f6695f0e772d27/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:07.271115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72ab8a-4f62-4fb7-a26d-b7f8dfb9d83b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.271400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72ab8a-4f62-4fb7-a26d-b7f8dfb9d83b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.271681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72ab8a-4f62-4fb7-a26d-b7f8dfb9d83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.297443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ef72ab8a-4f62-4fb7-a26d-b7f8dfb9d83b None None] MFA Rules not processed for user `5894cc21fe2b4c9eb4f6695f0e772d27`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:07.299158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a281cfbbaab44541b8e1d4fef6b86acf'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:07.299314 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ef72ab8a-4f62-4fb7-a26d-b7f8dfb9d83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:07.314298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64f966be-eb0f-443d-b765-9bb5f1c3c7a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a281cfbbaab44541b8e1d4fef6b86acf'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:07.324469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 976/1985] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:06 2022] DELETE /identity/v3/users/a281cfbbaab44541b8e1d4fef6b86acf => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:07.326096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d451d-023b-4d1f-8d53-40b95a357660 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.326501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d451d-023b-4d1f-8d53-40b95a357660 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.326786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d451d-023b-4d1f-8d53-40b95a357660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.344082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-193d451d-023b-4d1f-8d53-40b95a357660 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:07.345351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-193d451d-023b-4d1f-8d53-40b95a357660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:07.375388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1010/1986] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:07.380606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72ab8a-4f62-4fb7-a26d-b7f8dfb9d83b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:07.382013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ef72ab8a-4f62-4fb7-a26d-b7f8dfb9d83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:07.415392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 977/1987] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:07.425795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d451d-023b-4d1f-8d53-40b95a357660 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:07.427312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-193d451d-023b-4d1f-8d53-40b95a357660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:07.459990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72ab8a-4f62-4fb7-a26d-b7f8dfb9d83b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:07.465357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fca909-1554-4bfb-9852-e7b790b7299f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:07.466034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fca909-1554-4bfb-9852-e7b790b7299f admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.466322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fca909-1554-4bfb-9852-e7b790b7299f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.466601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fca909-1554-4bfb-9852-e7b790b7299f admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.467367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fca909-1554-4bfb-9852-e7b790b7299f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:07.469728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fca909-1554-4bfb-9852-e7b790b7299f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:07.491140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d451d-023b-4d1f-8d53-40b95a357660 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:07.495308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893818ec-c7db-41e5-a100-74388bf2c20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:07.496081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893818ec-c7db-41e5-a100-74388bf2c20e service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.496364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893818ec-c7db-41e5-a100-74388bf2c20e service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.496652 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893818ec-c7db-41e5-a100-74388bf2c20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.507347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893818ec-c7db-41e5-a100-74388bf2c20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:07.509235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893818ec-c7db-41e5-a100-74388bf2c20e service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:07.520273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 978/1988] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:07.590438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1011/1989] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:07 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:07.591895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893818ec-c7db-41e5-a100-74388bf2c20e service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:07.602293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893818ec-c7db-41e5-a100-74388bf2c20e service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:07.608809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:07.677392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20b64d-208b-4086-989b-8365ba08ee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:07.678622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20b64d-208b-4086-989b-8365ba08ee68 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.678910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20b64d-208b-4086-989b-8365ba08ee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.679194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20b64d-208b-4086-989b-8365ba08ee68 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.679896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20b64d-208b-4086-989b-8365ba08ee68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:07.681897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20b64d-208b-4086-989b-8365ba08ee68 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:07.794533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 979/1990] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:07 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:07.796034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fca909-1554-4bfb-9852-e7b790b7299f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:07.807966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fca909-1554-4bfb-9852-e7b790b7299f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:07.813199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be764a46-b587-41da-bf0d-d2935fd4bec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:07.813943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be764a46-b587-41da-bf0d-d2935fd4bec3 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.814359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be764a46-b587-41da-bf0d-d2935fd4bec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.814713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be764a46-b587-41da-bf0d-d2935fd4bec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.815450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be764a46-b587-41da-bf0d-d2935fd4bec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:07.817078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be764a46-b587-41da-bf0d-d2935fd4bec3 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:07.831022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1012/1991] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:07.844462 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20b64d-208b-4086-989b-8365ba08ee68 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:07.847397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20b64d-208b-4086-989b-8365ba08ee68 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:07.852557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026a31-7984-4c2e-a9ba-3241eb76db18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:07.853495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026a31-7984-4c2e-a9ba-3241eb76db18 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:07.853786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026a31-7984-4c2e-a9ba-3241eb76db18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:07.854069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026a31-7984-4c2e-a9ba-3241eb76db18 admin admin] PATH_INFO: `/v3/projects/197522cec0a641d99c94df0db0654d5b/users/99f58bc3d7d64e2b87e34922b7416e55/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:07.876451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026a31-7984-4c2e-a9ba-3241eb76db18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197522cec0a641d99c94df0db0654d5b, user_id=99f58bc3d7d64e2b87e34922b7416e55, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:07.879180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026a31-7984-4c2e-a9ba-3241eb76db18 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:07.957347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 980/1992] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:07 2022] PUT /identity/v3/projects/197522cec0a641d99c94df0db0654d5b/users/99f58bc3d7d64e2b87e34922b7416e55/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:07.961614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be764a46-b587-41da-bf0d-d2935fd4bec3 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:07.971584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be764a46-b587-41da-bf0d-d2935fd4bec3 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:08.035811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ceeba4-9890-4c49-ba01-d65b9d4b5d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:08.036491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ceeba4-9890-4c49-ba01-d65b9d4b5d5e admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:08.036833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ceeba4-9890-4c49-ba01-d65b9d4b5d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:08.037114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ceeba4-9890-4c49-ba01-d65b9d4b5d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:08.037844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ceeba4-9890-4c49-ba01-d65b9d4b5d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:08.039445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ceeba4-9890-4c49-ba01-d65b9d4b5d5e admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:08.049964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1013/1993] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:08.054709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026a31-7984-4c2e-a9ba-3241eb76db18 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:08.065674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ceeba4-9890-4c49-ba01-d65b9d4b5d5e admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:08.067439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e4ceeba4-9890-4c49-ba01-d65b9d4b5d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:08.069192 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026a31-7984-4c2e-a9ba-3241eb76db18 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:08.072400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:08.076373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86b038-7803-4d5b-969c-e3731b98e8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:08.077321 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86b038-7803-4d5b-969c-e3731b98e8e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:08.077611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86b038-7803-4d5b-969c-e3731b98e8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:08.077892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86b038-7803-4d5b-969c-e3731b98e8e3 admin admin] PATH_INFO: `/v3/projects/197522cec0a641d99c94df0db0654d5b/users/99f58bc3d7d64e2b87e34922b7416e55/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:08.080162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86b038-7803-4d5b-969c-e3731b98e8e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197522cec0a641d99c94df0db0654d5b, user_id=99f58bc3d7d64e2b87e34922b7416e55, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:08.083987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86b038-7803-4d5b-969c-e3731b98e8e3 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:08.089335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ceeba4-9890-4c49-ba01-d65b9d4b5d5e admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:08.096761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 981/1994] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:08 2022] PUT /identity/v3/projects/197522cec0a641d99c94df0db0654d5b/users/99f58bc3d7d64e2b87e34922b7416e55/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:08.107648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b9dfc-25a5-4d32-9316-3e2375c77ceb None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:08.107934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b9dfc-25a5-4d32-9316-3e2375c77ceb None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:08.108215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b9dfc-25a5-4d32-9316-3e2375c77ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:08.138271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-fa8b9dfc-25a5-4d32-9316-3e2375c77ceb None None] MFA Rules not processed for user `99f58bc3d7d64e2b87e34922b7416e55`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:08.139410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fa8b9dfc-25a5-4d32-9316-3e2375c77ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:08.163244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:08.164157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:08.164933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:08.168197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] PATH_INFO: `/v3/projects/db721455223c4f11addbdd66b5ed41a1` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:08.168197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db721455223c4f11addbdd66b5ed41a1)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:08.179472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:08.234701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 982/1995] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:08.262430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b9dfc-25a5-4d32-9316-3e2375c77ceb None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:08.266067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fa8b9dfc-25a5-4d32-9316-3e2375c77ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:08.331328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db721455223c4f11addbdd66b5ed41a1', 'request_id': 'req-f7f65c9e-d52d-45ee-b09d-725872376252'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.332013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b9dfc-25a5-4d32-9316-3e2375c77ceb None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:08.340600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db721455223c4f11addbdd66b5ed41a1', 'request_id': 'req-f7f65c9e-d52d-45ee-b09d-725872376252'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.347525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db721455223c4f11addbdd66b5ed41a1', 'request_id': 'req-f7f65c9e-d52d-45ee-b09d-725872376252'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.385054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7bb12-0c6e-41f1-a1db-74964c994e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:08.385714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7bb12-0c6e-41f1-a1db-74964c994e14 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:08.386004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7bb12-0c6e-41f1-a1db-74964c994e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:08.386287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7bb12-0c6e-41f1-a1db-74964c994e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:08.414858 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:08.419390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] The token cache is being invalidate because project db721455223c4f11addbdd66b5ed41a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:08.419690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.420595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.426494 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1014/1996] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:08 2022] DELETE /identity/v3/projects/db721455223c4f11addbdd66b5ed41a1 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:08.426494 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:08.428178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:08.449463 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7bb12-0c6e-41f1-a1db-74964c994e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:08.450915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7bb12-0c6e-41f1-a1db-74964c994e14 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:08.461372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 983/1997] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:08.494118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f65c9e-d52d-45ee-b09d-725872376252 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:08.499472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:08.500378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:08.500707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:08.500963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] PATH_INFO: `/v3/users/98e93f83bfcb432481e412f4e239eedd` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:08.502276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e93f83bfcb432481e412f4e239eedd)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:08.504211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:08.663862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:08.742132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e93f83bfcb432481e412f4e239eedd'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.765069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e93f83bfcb432481e412f4e239eedd'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.773302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e93f83bfcb432481e412f4e239eedd'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.774045 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e93f83bfcb432481e412f4e239eedd'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.864977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e234fdc3-069e-4c2e-b6fe-fd00441cf88a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e93f83bfcb432481e412f4e239eedd'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:08.872392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1015/1998] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:08 2022] DELETE /identity/v3/users/98e93f83bfcb432481e412f4e239eedd => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:09.139199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:09.301968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7bb12-0c6e-41f1-a1db-74964c994e14 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:09.316992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7bb12-0c6e-41f1-a1db-74964c994e14 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:09.394431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:09.395193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:09.395507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:09.395785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] PATH_INFO: `/v3/projects/f890f20b9e2c4f32a9ff87f8e1848cba` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:09.397657 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f890f20b9e2c4f32a9ff87f8e1848cba)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:09.399572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:09.501464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f890f20b9e2c4f32a9ff87f8e1848cba', 'request_id': 'req-64a5107f-7896-480e-adc4-d0f791aa6b8a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:09.508941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f890f20b9e2c4f32a9ff87f8e1848cba', 'request_id': 'req-64a5107f-7896-480e-adc4-d0f791aa6b8a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:09.578967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f890f20b9e2c4f32a9ff87f8e1848cba', 'request_id': 'req-64a5107f-7896-480e-adc4-d0f791aa6b8a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:09.591361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] The token cache is being invalidate because project f890f20b9e2c4f32a9ff87f8e1848cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:09.591643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:09.592286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-64a5107f-7896-480e-adc4-d0f791aa6b8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:09.593849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 984/1999] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:09 2022] DELETE /identity/v3/projects/f890f20b9e2c4f32a9ff87f8e1848cba => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:09.599040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d60d9a-f14b-4576-aeab-9a566e3e6e83 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:09.599541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d60d9a-f14b-4576-aeab-9a566e3e6e83 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:09.599830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d60d9a-f14b-4576-aeab-9a566e3e6e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:09.618839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-39d60d9a-f14b-4576-aeab-9a566e3e6e83 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:09.620050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-39d60d9a-f14b-4576-aeab-9a566e3e6e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:09.683910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1016/2000] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:09.691704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde5c3-681d-4461-8c4f-b251395ab94e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:09.692003 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde5c3-681d-4461-8c4f-b251395ab94e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:09.692286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde5c3-681d-4461-8c4f-b251395ab94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:09.715873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-06cde5c3-681d-4461-8c4f-b251395ab94e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:09.716900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-06cde5c3-681d-4461-8c4f-b251395ab94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:09.723136 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d60d9a-f14b-4576-aeab-9a566e3e6e83 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:09.724578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-39d60d9a-f14b-4576-aeab-9a566e3e6e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:09.725810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 985/2001] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:09.730048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cde5c3-681d-4461-8c4f-b251395ab94e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:09.741295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cde5c3-681d-4461-8c4f-b251395ab94e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:09.745479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:09.747680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014c8cb-0f07-4864-8e9a-d67453ff3ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:09.748480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014c8cb-0f07-4864-8e9a-d67453ff3ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:09.748813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014c8cb-0f07-4864-8e9a-d67453ff3ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:09.749115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014c8cb-0f07-4864-8e9a-d67453ff3ff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:09.754904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014c8cb-0f07-4864-8e9a-d67453ff3ff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:09.754904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014c8cb-0f07-4864-8e9a-d67453ff3ff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:09.754904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014c8cb-0f07-4864-8e9a-d67453ff3ff4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:09.764705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 986/2002] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:02:09.769151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014c8cb-0f07-4864-8e9a-d67453ff3ff4 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:09.783497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014c8cb-0f07-4864-8e9a-d67453ff3ff4 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:09.784412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4709460-ff81-4a8b-aaba-bf2c0f8b4006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:09.785109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4709460-ff81-4a8b-aaba-bf2c0f8b4006 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:09.785394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4709460-ff81-4a8b-aaba-bf2c0f8b4006 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:09.785672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4709460-ff81-4a8b-aaba-bf2c0f8b4006 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:09.786362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4709460-ff81-4a8b-aaba-bf2c0f8b4006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:09.786527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d60d9a-f14b-4576-aeab-9a566e3e6e83 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:09.788278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4709460-ff81-4a8b-aaba-bf2c0f8b4006 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:09.790786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677ca9b-4fd8-4373-8310-82e5d6217cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:09.791454 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677ca9b-4fd8-4373-8310-82e5d6217cba service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:09.791876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677ca9b-4fd8-4373-8310-82e5d6217cba service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:09.792185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677ca9b-4fd8-4373-8310-82e5d6217cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:09.794190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9677ca9b-4fd8-4373-8310-82e5d6217cba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:09.857183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677ca9b-4fd8-4373-8310-82e5d6217cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:09.858631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677ca9b-4fd8-4373-8310-82e5d6217cba service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:09.867533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1017/2003] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:09.914727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 987/2004] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:09 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:09.918717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677ca9b-4fd8-4373-8310-82e5d6217cba service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:09.928827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677ca9b-4fd8-4373-8310-82e5d6217cba service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:09.989389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460462f6-815e-4ca7-a4a1-aa3aaa46a03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:09.990827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460462f6-815e-4ca7-a4a1-aa3aaa46a03f admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:09.991123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460462f6-815e-4ca7-a4a1-aa3aaa46a03f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:09.991405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460462f6-815e-4ca7-a4a1-aa3aaa46a03f admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:09.992463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460462f6-815e-4ca7-a4a1-aa3aaa46a03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:09.994832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460462f6-815e-4ca7-a4a1-aa3aaa46a03f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:10.098931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1018/2005] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:09 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:10.103605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4709460-ff81-4a8b-aaba-bf2c0f8b4006 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:10.114175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4709460-ff81-4a8b-aaba-bf2c0f8b4006 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:10.118928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ed8aa-cbd4-481d-b5b5-4e80b6991197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:10.119579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed8aa-cbd4-481d-b5b5-4e80b6991197 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:10.119872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed8aa-cbd4-481d-b5b5-4e80b6991197 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:10.120128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed8aa-cbd4-481d-b5b5-4e80b6991197 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:10.120917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ed8aa-cbd4-481d-b5b5-4e80b6991197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:10.122505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ed8aa-cbd4-481d-b5b5-4e80b6991197 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:10.132792 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 988/2006] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:10.136994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460462f6-815e-4ca7-a4a1-aa3aaa46a03f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:10.147601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460462f6-815e-4ca7-a4a1-aa3aaa46a03f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:10.152359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bbf1f-4904-4bcc-b992-d0fa563ee6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:10.153086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bbf1f-4904-4bcc-b992-d0fa563ee6ff admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:10.159255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bbf1f-4904-4bcc-b992-d0fa563ee6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:10.159255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bbf1f-4904-4bcc-b992-d0fa563ee6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:10.159255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bbf1f-4904-4bcc-b992-d0fa563ee6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:10.159255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bbf1f-4904-4bcc-b992-d0fa563ee6ff admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:10.170804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1019/2007] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:10.171991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ed8aa-cbd4-481d-b5b5-4e80b6991197 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:10.182187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ed8aa-cbd4-481d-b5b5-4e80b6991197 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:10.186699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8701f3-a30f-4169-bf05-66cd5d1e92a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:10.188441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8701f3-a30f-4169-bf05-66cd5d1e92a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:10.188441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8701f3-a30f-4169-bf05-66cd5d1e92a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:10.188441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8701f3-a30f-4169-bf05-66cd5d1e92a9 admin admin] PATH_INFO: `/v3/projects/c3534114306a4599983b3d6389369ef3/users/1ffc8087dadc4290b7b9268cef1a4984/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:10.191472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:10.203285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8701f3-a30f-4169-bf05-66cd5d1e92a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3534114306a4599983b3d6389369ef3, user_id=1ffc8087dadc4290b7b9268cef1a4984, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:10.206047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8701f3-a30f-4169-bf05-66cd5d1e92a9 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:10.337172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 989/2008] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:10 2022] PUT /identity/v3/projects/c3534114306a4599983b3d6389369ef3/users/1ffc8087dadc4290b7b9268cef1a4984/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:10.339802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033046c-007f-4305-b56f-d0d90db84c5f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:10.340167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033046c-007f-4305-b56f-d0d90db84c5f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:10.340451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033046c-007f-4305-b56f-d0d90db84c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:10.368899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d033046c-007f-4305-b56f-d0d90db84c5f None None] MFA Rules not processed for user `1ffc8087dadc4290b7b9268cef1a4984`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:10.370231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d033046c-007f-4305-b56f-d0d90db84c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:10.412559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:10.453647 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1020/2009] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:10.458209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8701f3-a30f-4169-bf05-66cd5d1e92a9 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:10.538763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8701f3-a30f-4169-bf05-66cd5d1e92a9 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:10.543826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd40b08-9cbf-4e08-af3b-ab66dd7ada41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:10.544503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40b08-9cbf-4e08-af3b-ab66dd7ada41 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:10.545079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40b08-9cbf-4e08-af3b-ab66dd7ada41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:10.545361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40b08-9cbf-4e08-af3b-ab66dd7ada41 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:10.546050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd40b08-9cbf-4e08-af3b-ab66dd7ada41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:10.547979 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd40b08-9cbf-4e08-af3b-ab66dd7ada41 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:10.662576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 990/2010] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:10 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:10.663680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033046c-007f-4305-b56f-d0d90db84c5f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:10.675538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033046c-007f-4305-b56f-d0d90db84c5f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:10.741616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476a35e-00b6-441b-9da6-17499a73c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:10.742378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476a35e-00b6-441b-9da6-17499a73c3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:10.742743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476a35e-00b6-441b-9da6-17499a73c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:10.743092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476a35e-00b6-441b-9da6-17499a73c3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:10.744119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476a35e-00b6-441b-9da6-17499a73c3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:10.746443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476a35e-00b6-441b-9da6-17499a73c3c2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:10.847704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1021/2011] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:10 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:10.852355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd40b08-9cbf-4e08-af3b-ab66dd7ada41 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:10.863025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd40b08-9cbf-4e08-af3b-ab66dd7ada41 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:10.870766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802422e3-7db4-4490-9b16-3b126b678b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:10.870766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802422e3-7db4-4490-9b16-3b126b678b88 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:10.870766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802422e3-7db4-4490-9b16-3b126b678b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:10.870766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802422e3-7db4-4490-9b16-3b126b678b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:10.871302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802422e3-7db4-4490-9b16-3b126b678b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:10.872596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802422e3-7db4-4490-9b16-3b126b678b88 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:10.884125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 991/2012] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:10.888311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476a35e-00b6-441b-9da6-17499a73c3c2 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:10.899074 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476a35e-00b6-441b-9da6-17499a73c3c2 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:10.903673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f961-59f7-49ab-b475-c63100282fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:10.904550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f961-59f7-49ab-b475-c63100282fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:10.904934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f961-59f7-49ab-b475-c63100282fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:10.905241 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f961-59f7-49ab-b475-c63100282fcb admin admin] PATH_INFO: `/v3/projects/db727c61d76449118493687e55e231d4/users/0d82fabd3e444471a6d2aabd7b2fa20a/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:10.920767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980f961-59f7-49ab-b475-c63100282fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=db727c61d76449118493687e55e231d4, user_id=0d82fabd3e444471a6d2aabd7b2fa20a, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:10.923371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980f961-59f7-49ab-b475-c63100282fcb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:10.996178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1022/2013] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:10 2022] PUT /identity/v3/projects/db727c61d76449118493687e55e231d4/users/0d82fabd3e444471a6d2aabd7b2fa20a/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:11.000370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802422e3-7db4-4490-9b16-3b126b678b88 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:11.010537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802422e3-7db4-4490-9b16-3b126b678b88 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:11.075688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81025567-6400-46ef-8fb4-e1382093cbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:11.076372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81025567-6400-46ef-8fb4-e1382093cbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:11.076681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81025567-6400-46ef-8fb4-e1382093cbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:11.076971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81025567-6400-46ef-8fb4-e1382093cbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:11.077738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81025567-6400-46ef-8fb4-e1382093cbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:11.079336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81025567-6400-46ef-8fb4-e1382093cbf0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:11.089419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 992/2014] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:11.093503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f961-59f7-49ab-b475-c63100282fcb admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:11.104604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f961-59f7-49ab-b475-c63100282fcb admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:11.109340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0965fdb-703a-486d-bbcc-e06c30cdbb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:11.110205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0965fdb-703a-486d-bbcc-e06c30cdbb62 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:11.110553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0965fdb-703a-486d-bbcc-e06c30cdbb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:11.110764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0965fdb-703a-486d-bbcc-e06c30cdbb62 admin admin] PATH_INFO: `/v3/projects/db727c61d76449118493687e55e231d4/users/0d82fabd3e444471a6d2aabd7b2fa20a/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:11.112744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0965fdb-703a-486d-bbcc-e06c30cdbb62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db727c61d76449118493687e55e231d4, user_id=0d82fabd3e444471a6d2aabd7b2fa20a, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:11.116083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0965fdb-703a-486d-bbcc-e06c30cdbb62 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:11.126688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1023/2015] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:11 2022] PUT /identity/v3/projects/db727c61d76449118493687e55e231d4/users/0d82fabd3e444471a6d2aabd7b2fa20a/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:11.140867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181f52a-f5de-4a7d-9cad-010b62eab047 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:11.141183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181f52a-f5de-4a7d-9cad-010b62eab047 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:11.141510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181f52a-f5de-4a7d-9cad-010b62eab047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:11.168379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f181f52a-f5de-4a7d-9cad-010b62eab047 None None] MFA Rules not processed for user `0d82fabd3e444471a6d2aabd7b2fa20a`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:11.169625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f181f52a-f5de-4a7d-9cad-010b62eab047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:11.213162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:11.237446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 993/2016] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:11.249186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0965fdb-703a-486d-bbcc-e06c30cdbb62 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:11.305817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0965fdb-703a-486d-bbcc-e06c30cdbb62 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:11.309433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791a321-9129-47ca-9286-f931388cd703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:11.310084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791a321-9129-47ca-9286-f931388cd703 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:11.310367 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791a321-9129-47ca-9286-f931388cd703 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:11.310642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791a321-9129-47ca-9286-f931388cd703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:11.353849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:11.365624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791a321-9129-47ca-9286-f931388cd703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:11.367061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791a321-9129-47ca-9286-f931388cd703 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:11.376533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1024/2017] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:20.601783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181f52a-f5de-4a7d-9cad-010b62eab047 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:20.603729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f181f52a-f5de-4a7d-9cad-010b62eab047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:20.683194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181f52a-f5de-4a7d-9cad-010b62eab047 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:20.688456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:20.689431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:20.693635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:20.693635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] PATH_INFO: `/v3/users/c359517bc5904f85919773f3d6816fe5` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:20.693635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c359517bc5904f85919773f3d6816fe5)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:20.693635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:20.905690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c359517bc5904f85919773f3d6816fe5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:20.912905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c359517bc5904f85919773f3d6816fe5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:20.913697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c359517bc5904f85919773f3d6816fe5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:20.978794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c359517bc5904f85919773f3d6816fe5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:20.991653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c359517bc5904f85919773f3d6816fe5'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:20.997931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 994/2018] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:20 2022] DELETE /identity/v3/users/c359517bc5904f85919773f3d6816fe5 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:21.002440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f938bf-a175-4f9b-94e0-e743186fa89e None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:21.002923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f938bf-a175-4f9b-94e0-e743186fa89e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:21.003119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f938bf-a175-4f9b-94e0-e743186fa89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:21.020620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-91f938bf-a175-4f9b-94e0-e743186fa89e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:21.021864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-91f938bf-a175-4f9b-94e0-e743186fa89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:21.084159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1025/2019] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:21.094879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:21.096428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:21.162473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e2970-0f6a-4986-977d-030ceb13156f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:21.167832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39c46f-558d-4ae7-8dd0-c23d0d731a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:21.171578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c46f-558d-4ae7-8dd0-c23d0d731a79 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:21.171578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c46f-558d-4ae7-8dd0-c23d0d731a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:21.171578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c46f-558d-4ae7-8dd0-c23d0d731a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:21.184082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39c46f-558d-4ae7-8dd0-c23d0d731a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:21.185789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39c46f-558d-4ae7-8dd0-c23d0d731a79 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:21.199614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 995/2020] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:21.776352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f938bf-a175-4f9b-94e0-e743186fa89e None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:21.778284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-91f938bf-a175-4f9b-94e0-e743186fa89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:21.794965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f938bf-a175-4f9b-94e0-e743186fa89e None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:21.814644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:21.818888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:21.819536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:21.819972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] PATH_INFO: `/v3/projects/58e7d07a9c8a4543942ef788fef8fba1` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:21.824007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58e7d07a9c8a4543942ef788fef8fba1)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:21.826979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:21.989893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58e7d07a9c8a4543942ef788fef8fba1', 'request_id': 'req-81515f99-9929-4c34-bd43-6471a8ce7107'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:21.999006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58e7d07a9c8a4543942ef788fef8fba1', 'request_id': 'req-81515f99-9929-4c34-bd43-6471a8ce7107'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:22.007736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58e7d07a9c8a4543942ef788fef8fba1', 'request_id': 'req-81515f99-9929-4c34-bd43-6471a8ce7107'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:22.099697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] The token cache is being invalidate because project 58e7d07a9c8a4543942ef788fef8fba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:22.101574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:22.102177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:22.104133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1026/2021] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:21 2022] DELETE /identity/v3/projects/58e7d07a9c8a4543942ef788fef8fba1 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:22.108265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39c46f-558d-4ae7-8dd0-c23d0d731a79 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:22.113834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:22.122749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39c46f-558d-4ae7-8dd0-c23d0d731a79 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:22.220467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:22.221741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:22.222264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:22.222825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] PATH_INFO: `/v3/users/50b135d868884b87a98ca54911596936` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:22.224182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b135d868884b87a98ca54911596936)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:22.227249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:22.516081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b135d868884b87a98ca54911596936'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:22.524833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b135d868884b87a98ca54911596936'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:22.525746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b135d868884b87a98ca54911596936'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:22.671482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b135d868884b87a98ca54911596936'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:22.706459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b135d868884b87a98ca54911596936'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:22.714590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 996/2022] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:22 2022] DELETE /identity/v3/users/50b135d868884b87a98ca54911596936 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:23.202882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:23.203436 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:23.206166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:23.208316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:23.309565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcff187-730c-4b91-8d0e-0d679622b7c2 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:23.311958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81515f99-9929-4c34-bd43-6471a8ce7107 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:23.315697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:23.316902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:23.317512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:23.317862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] PATH_INFO: `/v3/projects/7df1cbc1c4d4475a89afee39870ccc22` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:23.320055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df1cbc1c4d4475a89afee39870ccc22)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:23.322738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:23.347095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:23.364436 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:23.375561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc17f0c-df71-4b62-9268-6a7e0543abb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:23.377782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc17f0c-df71-4b62-9268-6a7e0543abb8 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:23.377782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc17f0c-df71-4b62-9268-6a7e0543abb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:23.377782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc17f0c-df71-4b62-9268-6a7e0543abb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:23.378980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2cc17f0c-df71-4b62-9268-6a7e0543abb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:23.447593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc17f0c-df71-4b62-9268-6a7e0543abb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:23.449096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc17f0c-df71-4b62-9268-6a7e0543abb8 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:23.458874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1027/2023] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:23.515742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df1cbc1c4d4475a89afee39870ccc22', 'request_id': 'req-83872a91-4949-4830-85d4-1a00e19a3d1a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:23.522918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df1cbc1c4d4475a89afee39870ccc22', 'request_id': 'req-83872a91-4949-4830-85d4-1a00e19a3d1a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:23.617205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df1cbc1c4d4475a89afee39870ccc22', 'request_id': 'req-83872a91-4949-4830-85d4-1a00e19a3d1a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:23.630071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] The token cache is being invalidate because project 7df1cbc1c4d4475a89afee39870ccc22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:23.630369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:23.631031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:23.632760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 997/2024] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:23 2022] DELETE /identity/v3/projects/7df1cbc1c4d4475a89afee39870ccc22 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:23.637909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c23df1-7ca8-40b8-9702-eb4d62e91896 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:23.638283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c23df1-7ca8-40b8-9702-eb4d62e91896 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:23.638573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c23df1-7ca8-40b8-9702-eb4d62e91896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:23.649028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:23.650601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:23.658732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-64c23df1-7ca8-40b8-9702-eb4d62e91896 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:23.659922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-64c23df1-7ca8-40b8-9702-eb4d62e91896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:23.707418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83872a91-4949-4830-85d4-1a00e19a3d1a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:23.711751 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a946dd5-5252-47ed-be61-a96615203063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:23.712410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a946dd5-5252-47ed-be61-a96615203063 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:23.712877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a946dd5-5252-47ed-be61-a96615203063 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:23.713055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a946dd5-5252-47ed-be61-a96615203063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:23.715450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6a946dd5-5252-47ed-be61-a96615203063 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:23.728958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1028/2025] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:23.736504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def61539-422f-425d-b8d5-1f89d680e5ba None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:23.736899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def61539-422f-425d-b8d5-1f89d680e5ba None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:23.737428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def61539-422f-425d-b8d5-1f89d680e5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:23.759540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-def61539-422f-425d-b8d5-1f89d680e5ba None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:23.763275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-def61539-422f-425d-b8d5-1f89d680e5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:23.775422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1029/2026] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:23.776632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def61539-422f-425d-b8d5-1f89d680e5ba None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:23.789412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def61539-422f-425d-b8d5-1f89d680e5ba None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:23.794154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f06499-a39d-49d4-bd46-3971672f2e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:23.795184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f06499-a39d-49d4-bd46-3971672f2e4d admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:23.795498 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f06499-a39d-49d4-bd46-3971672f2e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:23.795774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f06499-a39d-49d4-bd46-3971672f2e4d admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:23.796506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f06499-a39d-49d4-bd46-3971672f2e4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:23.796900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f06499-a39d-49d4-bd46-3971672f2e4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:23.798496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f06499-a39d-49d4-bd46-3971672f2e4d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:23.802150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a946dd5-5252-47ed-be61-a96615203063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:23.803899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a946dd5-5252-47ed-be61-a96615203063 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:23.814566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1030/2027] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:02:23.814780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 998/2028] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:23.815680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f06499-a39d-49d4-bd46-3971672f2e4d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:23.826750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f06499-a39d-49d4-bd46-3971672f2e4d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:23.833470 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcbf13-3a07-4b5b-b2a9-4ecb33249b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:23.844513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcbf13-3a07-4b5b-b2a9-4ecb33249b5b admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:23.844513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcbf13-3a07-4b5b-b2a9-4ecb33249b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:23.844513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcbf13-3a07-4b5b-b2a9-4ecb33249b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:23.844513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcbf13-3a07-4b5b-b2a9-4ecb33249b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:23.844513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcbf13-3a07-4b5b-b2a9-4ecb33249b5b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:23.920141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1031/2029] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:23 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:23.923988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a946dd5-5252-47ed-be61-a96615203063 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:23.934607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a946dd5-5252-47ed-be61-a96615203063 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:23.991300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aeba3-fbb6-4e9c-8931-6524e964e046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:23.992112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aeba3-fbb6-4e9c-8931-6524e964e046 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:23.992399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aeba3-fbb6-4e9c-8931-6524e964e046 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:23.992727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aeba3-fbb6-4e9c-8931-6524e964e046 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:23.994162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060aeba3-fbb6-4e9c-8931-6524e964e046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:23.996218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060aeba3-fbb6-4e9c-8931-6524e964e046 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:24.008999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:24.127856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 999/2030] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:23 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:24.132325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcbf13-3a07-4b5b-b2a9-4ecb33249b5b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:24.142800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcbf13-3a07-4b5b-b2a9-4ecb33249b5b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:24.148008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a010e6-37b3-4414-8ec2-458888add0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:24.148734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a010e6-37b3-4414-8ec2-458888add0ef admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:24.148982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a010e6-37b3-4414-8ec2-458888add0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:24.149255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a010e6-37b3-4414-8ec2-458888add0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:24.149984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a010e6-37b3-4414-8ec2-458888add0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:24.151569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a010e6-37b3-4414-8ec2-458888add0ef admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:24.161864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1032/2031] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:24.165838 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aeba3-fbb6-4e9c-8931-6524e964e046 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:24.176318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aeba3-fbb6-4e9c-8931-6524e964e046 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:24.181543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954d6b5-e863-46c9-a35f-41732a566b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:24.182278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954d6b5-e863-46c9-a35f-41732a566b44 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:24.182536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954d6b5-e863-46c9-a35f-41732a566b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:24.182815 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954d6b5-e863-46c9-a35f-41732a566b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:24.183546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954d6b5-e863-46c9-a35f-41732a566b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:24.185186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954d6b5-e863-46c9-a35f-41732a566b44 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:24.196397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1000/2032] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:24.200539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a010e6-37b3-4414-8ec2-458888add0ef admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:24.210530 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a010e6-37b3-4414-8ec2-458888add0ef admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:24.215140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dcfaf-7157-419d-bf86-3505cf066648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:24.216019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dcfaf-7157-419d-bf86-3505cf066648 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:24.216311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dcfaf-7157-419d-bf86-3505cf066648 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:24.216696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dcfaf-7157-419d-bf86-3505cf066648 admin admin] PATH_INFO: `/v3/projects/5b7b51189fd94b8b82335839496687b4/users/72fdf7a225484fef952e5d1388da5ec3/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:24.231431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dcfaf-7157-419d-bf86-3505cf066648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7b51189fd94b8b82335839496687b4, user_id=72fdf7a225484fef952e5d1388da5ec3, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:24.234110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dcfaf-7157-419d-bf86-3505cf066648 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:24.379142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:24.379142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1033/2033] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:24 2022] PUT /identity/v3/projects/5b7b51189fd94b8b82335839496687b4/users/72fdf7a225484fef952e5d1388da5ec3/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:24.389798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a2bf5-34e2-4697-b5e0-f49fa2c9eb5a None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:24.390292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a2bf5-34e2-4697-b5e0-f49fa2c9eb5a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:24.390423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a2bf5-34e2-4697-b5e0-f49fa2c9eb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:24.421853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-289a2bf5-34e2-4697-b5e0-f49fa2c9eb5a None None] MFA Rules not processed for user `72fdf7a225484fef952e5d1388da5ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:24.423091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-289a2bf5-34e2-4697-b5e0-f49fa2c9eb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:24.486692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1001/2034] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:24.492024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dcfaf-7157-419d-bf86-3505cf066648 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:24.561518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dcfaf-7157-419d-bf86-3505cf066648 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:24.564172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ff275-bf0c-4811-ade7-6e4d3802812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:24.564920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ff275-bf0c-4811-ade7-6e4d3802812b admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:24.565201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ff275-bf0c-4811-ade7-6e4d3802812b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:24.565477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ff275-bf0c-4811-ade7-6e4d3802812b admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:24.566270 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ff275-bf0c-4811-ade7-6e4d3802812b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:24.568265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ff275-bf0c-4811-ade7-6e4d3802812b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:24.637531 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1034/2035] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:24 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:24.638759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a2bf5-34e2-4697-b5e0-f49fa2c9eb5a None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:24.650793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a2bf5-34e2-4697-b5e0-f49fa2c9eb5a None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:24.710669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aa03e-8c83-4532-9068-343ed8b58d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:24.711361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aa03e-8c83-4532-9068-343ed8b58d69 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:24.711686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aa03e-8c83-4532-9068-343ed8b58d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:24.711957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aa03e-8c83-4532-9068-343ed8b58d69 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:24.712660 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2aa03e-8c83-4532-9068-343ed8b58d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:24.716740 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2aa03e-8c83-4532-9068-343ed8b58d69 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:24.773466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1002/2036] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:24 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:24.777455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ff275-bf0c-4811-ade7-6e4d3802812b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:24.788839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ff275-bf0c-4811-ade7-6e4d3802812b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:24.793630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b98237-9d1f-46a0-a95c-2cc520d10a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:24.794309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b98237-9d1f-46a0-a95c-2cc520d10a6f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:24.794580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b98237-9d1f-46a0-a95c-2cc520d10a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:24.795160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b98237-9d1f-46a0-a95c-2cc520d10a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:24.795916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b98237-9d1f-46a0-a95c-2cc520d10a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:24.797602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b98237-9d1f-46a0-a95c-2cc520d10a6f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:24.807874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1035/2037] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:24.812238 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aa03e-8c83-4532-9068-343ed8b58d69 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:24.823349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aa03e-8c83-4532-9068-343ed8b58d69 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:24.828447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b9272-b2b8-4b00-a119-2651244a5868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:24.829367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b9272-b2b8-4b00-a119-2651244a5868 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:24.829653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b9272-b2b8-4b00-a119-2651244a5868 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:24.829934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b9272-b2b8-4b00-a119-2651244a5868 admin admin] PATH_INFO: `/v3/projects/f84c1139185e4314b313fced73ddc8d4/users/1fbce5b046304c338256602e5e92ac62/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:24.846475 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b9272-b2b8-4b00-a119-2651244a5868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84c1139185e4314b313fced73ddc8d4, user_id=1fbce5b046304c338256602e5e92ac62, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:24.849223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b9272-b2b8-4b00-a119-2651244a5868 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:24.910383 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1003/2038] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:24 2022] PUT /identity/v3/projects/f84c1139185e4314b313fced73ddc8d4/users/1fbce5b046304c338256602e5e92ac62/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:24.914645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b98237-9d1f-46a0-a95c-2cc520d10a6f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:24.925066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b98237-9d1f-46a0-a95c-2cc520d10a6f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:24.982526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b0dcf-4982-43d4-9ef9-4c23f262c76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:24.983211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b0dcf-4982-43d4-9ef9-4c23f262c76c admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:24.983501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b0dcf-4982-43d4-9ef9-4c23f262c76c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:24.983780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b0dcf-4982-43d4-9ef9-4c23f262c76c admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:24.984522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b0dcf-4982-43d4-9ef9-4c23f262c76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:24.986230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b0dcf-4982-43d4-9ef9-4c23f262c76c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:24.997610 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1036/2039] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:25.001715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b9272-b2b8-4b00-a119-2651244a5868 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:25.013062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b9272-b2b8-4b00-a119-2651244a5868 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:25.017985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3126f8-b38d-450c-adac-1089de00efcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:25.018880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3126f8-b38d-450c-adac-1089de00efcf admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:25.019168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3126f8-b38d-450c-adac-1089de00efcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:25.019448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3126f8-b38d-450c-adac-1089de00efcf admin admin] PATH_INFO: `/v3/projects/f84c1139185e4314b313fced73ddc8d4/users/1fbce5b046304c338256602e5e92ac62/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:25.021452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3126f8-b38d-450c-adac-1089de00efcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84c1139185e4314b313fced73ddc8d4, user_id=1fbce5b046304c338256602e5e92ac62, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:25.024096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3126f8-b38d-450c-adac-1089de00efcf admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:25.035137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1004/2040] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:25 2022] PUT /identity/v3/projects/f84c1139185e4314b313fced73ddc8d4/users/1fbce5b046304c338256602e5e92ac62/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:25.045587 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c3104-bdf7-40c0-9e96-fc80e9996f70 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:25.045887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c3104-bdf7-40c0-9e96-fc80e9996f70 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:25.046165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c3104-bdf7-40c0-9e96-fc80e9996f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:25.074112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-4d2c3104-bdf7-40c0-9e96-fc80e9996f70 None None] MFA Rules not processed for user `1fbce5b046304c338256602e5e92ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:25.077216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4d2c3104-bdf7-40c0-9e96-fc80e9996f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:25.136964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:25.147657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1037/2041] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:25.159805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3126f8-b38d-450c-adac-1089de00efcf admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:25.161547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2b3126f8-b38d-450c-adac-1089de00efcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:25.196734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:25.223345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3126f8-b38d-450c-adac-1089de00efcf admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:25.227285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73dcd8b-20fa-4f76-9c54-84a44a27c73a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:25.231896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dcd8b-20fa-4f76-9c54-84a44a27c73a service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:25.231896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dcd8b-20fa-4f76-9c54-84a44a27c73a service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:25.231896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dcd8b-20fa-4f76-9c54-84a44a27c73a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:25.238049 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73dcd8b-20fa-4f76-9c54-84a44a27c73a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:25.239507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73dcd8b-20fa-4f76-9c54-84a44a27c73a service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:25.249501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1005/2042] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:25.458509 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c3104-bdf7-40c0-9e96-fc80e9996f70 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:25.468025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c3104-bdf7-40c0-9e96-fc80e9996f70 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:25.471588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c76e6-4e8c-489a-9a2b-50b26bb774bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:25.472234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c76e6-4e8c-489a-9a2b-50b26bb774bf service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:25.472523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c76e6-4e8c-489a-9a2b-50b26bb774bf service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:25.472836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c76e6-4e8c-489a-9a2b-50b26bb774bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:25.528156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c76e6-4e8c-489a-9a2b-50b26bb774bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:25.529712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c76e6-4e8c-489a-9a2b-50b26bb774bf service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:25.539222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1038/2043] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:26.049832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73dcd8b-20fa-4f76-9c54-84a44a27c73a service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:26.113001 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73dcd8b-20fa-4f76-9c54-84a44a27c73a service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:26.118298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:26.119182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:26.119574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:26.119857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] PATH_INFO: `/v3/users/72fdf7a225484fef952e5d1388da5ec3` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:26.121099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72fdf7a225484fef952e5d1388da5ec3)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:26.123087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:26.289674 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72fdf7a225484fef952e5d1388da5ec3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:26.297912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72fdf7a225484fef952e5d1388da5ec3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:26.298648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72fdf7a225484fef952e5d1388da5ec3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:26.338017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72fdf7a225484fef952e5d1388da5ec3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:26.352414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72fdf7a225484fef952e5d1388da5ec3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:26.359298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1006/2044] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:26 2022] DELETE /identity/v3/users/72fdf7a225484fef952e5d1388da5ec3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:26.364163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c34354-e4ab-415d-86dd-9e8ee9e5ca98 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:26.364494 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c34354-e4ab-415d-86dd-9e8ee9e5ca98 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:26.367539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c34354-e4ab-415d-86dd-9e8ee9e5ca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:26.383190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-69c34354-e4ab-415d-86dd-9e8ee9e5ca98 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:26.384770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-69c34354-e4ab-415d-86dd-9e8ee9e5ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:26.448591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1039/2045] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:26.458263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:26.459959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:26.529914 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:26.534939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2396c4-2a21-456e-bb6e-406d5c0b2329 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:26.538562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7726c-fcb9-43bc-89de-1a4210d9441e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:26.539216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7726c-fcb9-43bc-89de-1a4210d9441e service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:26.539500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7726c-fcb9-43bc-89de-1a4210d9441e service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:26.539783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7726c-fcb9-43bc-89de-1a4210d9441e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:26.549576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7726c-fcb9-43bc-89de-1a4210d9441e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:26.550998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7726c-fcb9-43bc-89de-1a4210d9441e service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:26.561506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1007/2046] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:27.059758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c34354-e4ab-415d-86dd-9e8ee9e5ca98 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:27.059758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-69c34354-e4ab-415d-86dd-9e8ee9e5ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:27.069802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c34354-e4ab-415d-86dd-9e8ee9e5ca98 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:27.075029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:27.075868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:27.076151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:27.076621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] PATH_INFO: `/v3/projects/5b7b51189fd94b8b82335839496687b4` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:27.078197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7b51189fd94b8b82335839496687b4)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:27.080399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:27.292880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7b51189fd94b8b82335839496687b4', 'request_id': 'req-ce60e56e-c192-4518-a3da-2835a33055af'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.300073 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7b51189fd94b8b82335839496687b4', 'request_id': 'req-ce60e56e-c192-4518-a3da-2835a33055af'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.306688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7b51189fd94b8b82335839496687b4', 'request_id': 'req-ce60e56e-c192-4518-a3da-2835a33055af'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.406535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] The token cache is being invalidate because project 5b7b51189fd94b8b82335839496687b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:27.410712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.410712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.412852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1040/2047] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:27 2022] DELETE /identity/v3/projects/5b7b51189fd94b8b82335839496687b4 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:27.414327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7726c-fcb9-43bc-89de-1a4210d9441e service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:27.424698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7726c-fcb9-43bc-89de-1a4210d9441e service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:27.496524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:27.497487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:27.497768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:27.498045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] PATH_INFO: `/v3/users/1fbce5b046304c338256602e5e92ac62` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:27.499235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fbce5b046304c338256602e5e92ac62)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:27.501211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:27.774270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fbce5b046304c338256602e5e92ac62'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.781504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fbce5b046304c338256602e5e92ac62'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.782181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fbce5b046304c338256602e5e92ac62'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.846917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fbce5b046304c338256602e5e92ac62'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.870776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6a4630b3-260c-414a-8324-27ee569f94ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fbce5b046304c338256602e5e92ac62'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:27.877552 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1008/2048] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:27 2022] DELETE /identity/v3/users/1fbce5b046304c338256602e5e92ac62 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:28.557648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:28.565392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:28.585983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60e56e-c192-4518-a3da-2835a33055af admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:28.606724 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:28.674219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:28.675331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:28.676355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:28.676355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] PATH_INFO: `/v3/projects/f84c1139185e4314b313fced73ddc8d4` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:28.678084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84c1139185e4314b313fced73ddc8d4)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:28.680699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:28.808556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84c1139185e4314b313fced73ddc8d4', 'request_id': 'req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:28.814143 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84c1139185e4314b313fced73ddc8d4', 'request_id': 'req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:28.822027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84c1139185e4314b313fced73ddc8d4', 'request_id': 'req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:28.945869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] The token cache is being invalidate because project f84c1139185e4314b313fced73ddc8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:28.946081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:28.946702 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-5dd85e38-8600-4e4a-a787-d9b4560a7f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:28.948322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1041/2049] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:28 2022] DELETE /identity/v3/projects/f84c1139185e4314b313fced73ddc8d4 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:28.953866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aedc6b-9072-4179-900b-c073ddae56f4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:28.954347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aedc6b-9072-4179-900b-c073ddae56f4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:28.954465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aedc6b-9072-4179-900b-c073ddae56f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:28.976794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-98aedc6b-9072-4179-900b-c073ddae56f4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:28.976794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-98aedc6b-9072-4179-900b-c073ddae56f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:29.036882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1009/2050] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:29.045597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b431a6f-bbc8-4e43-b1f3-3e08839d0330 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:29.045886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b431a6f-bbc8-4e43-b1f3-3e08839d0330 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:29.046170 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b431a6f-bbc8-4e43-b1f3-3e08839d0330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:29.064350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1b431a6f-bbc8-4e43-b1f3-3e08839d0330 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:29.065728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1b431a6f-bbc8-4e43-b1f3-3e08839d0330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:29.074043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1042/2051] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:29.078305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aedc6b-9072-4179-900b-c073ddae56f4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:29.089160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aedc6b-9072-4179-900b-c073ddae56f4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:29.094714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45506a35-9f2f-438b-8688-8ddddbce619f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:29.095351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45506a35-9f2f-438b-8688-8ddddbce619f admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:29.095639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45506a35-9f2f-438b-8688-8ddddbce619f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:29.095922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45506a35-9f2f-438b-8688-8ddddbce619f admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:29.096775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45506a35-9f2f-438b-8688-8ddddbce619f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:29.097155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45506a35-9f2f-438b-8688-8ddddbce619f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:29.098763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45506a35-9f2f-438b-8688-8ddddbce619f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:29.106430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:29.112313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1010/2052] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:02:29.116689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b431a6f-bbc8-4e43-b1f3-3e08839d0330 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:29.126181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b431a6f-bbc8-4e43-b1f3-3e08839d0330 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:29.131177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3f4b4-9a65-4174-bf26-d2a85587fb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:29.132358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3f4b4-9a65-4174-bf26-d2a85587fb6f admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:29.132671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3f4b4-9a65-4174-bf26-d2a85587fb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:29.132969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3f4b4-9a65-4174-bf26-d2a85587fb6f admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:29.133734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3f4b4-9a65-4174-bf26-d2a85587fb6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:29.135672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3f4b4-9a65-4174-bf26-d2a85587fb6f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:29.212692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1043/2053] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:29 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:29.214137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45506a35-9f2f-438b-8688-8ddddbce619f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:29.223952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45506a35-9f2f-438b-8688-8ddddbce619f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:29.282910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05508ca-b9fc-4e91-a3d8-35f640a42f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:29.283590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05508ca-b9fc-4e91-a3d8-35f640a42f17 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:29.283877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05508ca-b9fc-4e91-a3d8-35f640a42f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:29.284159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05508ca-b9fc-4e91-a3d8-35f640a42f17 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:29.284957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05508ca-b9fc-4e91-a3d8-35f640a42f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:29.297073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05508ca-b9fc-4e91-a3d8-35f640a42f17 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:29.377972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1011/2054] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:29 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:29.379227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3f4b4-9a65-4174-bf26-d2a85587fb6f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:29.395938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3f4b4-9a65-4174-bf26-d2a85587fb6f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:29.401753 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39dfb-2011-4322-962b-37244e08a855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:29.402420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39dfb-2011-4322-962b-37244e08a855 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:29.405736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39dfb-2011-4322-962b-37244e08a855 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:29.405736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39dfb-2011-4322-962b-37244e08a855 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:29.405736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b39dfb-2011-4322-962b-37244e08a855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:29.410132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b39dfb-2011-4322-962b-37244e08a855 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:29.426327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1044/2055] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:29.434386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05508ca-b9fc-4e91-a3d8-35f640a42f17 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:29.444099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05508ca-b9fc-4e91-a3d8-35f640a42f17 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:29.453846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b8721-54b6-4c62-9677-381f1e70bd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:29.455879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b8721-54b6-4c62-9677-381f1e70bd88 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:29.456297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b8721-54b6-4c62-9677-381f1e70bd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:29.456852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b8721-54b6-4c62-9677-381f1e70bd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:29.463045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b8721-54b6-4c62-9677-381f1e70bd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:29.463045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b8721-54b6-4c62-9677-381f1e70bd88 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:29.483987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1012/2056] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:29.485047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39dfb-2011-4322-962b-37244e08a855 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:29.495394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39dfb-2011-4322-962b-37244e08a855 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:29.500136 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ca9d3-9eaa-4bd4-b1e6-5f55934314d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:29.501063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ca9d3-9eaa-4bd4-b1e6-5f55934314d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:29.501349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ca9d3-9eaa-4bd4-b1e6-5f55934314d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:29.501626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ca9d3-9eaa-4bd4-b1e6-5f55934314d8 admin admin] PATH_INFO: `/v3/projects/557563d484864d6f870d40dafcca08e1/users/bc76fd7a5bf24013a2d382657a29d4de/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:29.516500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ca9d3-9eaa-4bd4-b1e6-5f55934314d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557563d484864d6f870d40dafcca08e1, user_id=bc76fd7a5bf24013a2d382657a29d4de, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:29.519198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ca9d3-9eaa-4bd4-b1e6-5f55934314d8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:29.641612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1045/2057] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:29 2022] PUT /identity/v3/projects/557563d484864d6f870d40dafcca08e1/users/bc76fd7a5bf24013a2d382657a29d4de/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:29.651589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5032d7-44d9-4e0b-978d-61d1cd6343a0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:29.651998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5032d7-44d9-4e0b-978d-61d1cd6343a0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:29.652356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5032d7-44d9-4e0b-978d-61d1cd6343a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:29.673659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3e5032d7-44d9-4e0b-978d-61d1cd6343a0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:29.675245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3e5032d7-44d9-4e0b-978d-61d1cd6343a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:29.752356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1013/2058] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:29.770803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ca9d3-9eaa-4bd4-b1e6-5f55934314d8 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:29.773606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-101ca9d3-9eaa-4bd4-b1e6-5f55934314d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:29.840810 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:29.860550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ca9d3-9eaa-4bd4-b1e6-5f55934314d8 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:29.860550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b47947-c58b-4cef-a4ba-63d62a3c7301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:29.860550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b47947-c58b-4cef-a4ba-63d62a3c7301 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:29.860550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b47947-c58b-4cef-a4ba-63d62a3c7301 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:29.860923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b47947-c58b-4cef-a4ba-63d62a3c7301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:29.869422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b47947-c58b-4cef-a4ba-63d62a3c7301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:29.874474 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b47947-c58b-4cef-a4ba-63d62a3c7301 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:29.881499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1046/2059] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:30.105366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5032d7-44d9-4e0b-978d-61d1cd6343a0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:30.106923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3e5032d7-44d9-4e0b-978d-61d1cd6343a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:30.121070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5032d7-44d9-4e0b-978d-61d1cd6343a0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:30.126069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:30.127129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:30.127432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:30.136810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] PATH_INFO: `/v3/users/850e54d515fd4233bbc46fc72ab41812` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:30.136810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] RBAC: Authorizing `identity:delete_user(user_id=850e54d515fd4233bbc46fc72ab41812)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:30.136810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:30.442532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '850e54d515fd4233bbc46fc72ab41812'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:30.452965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '850e54d515fd4233bbc46fc72ab41812'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:30.454272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '850e54d515fd4233bbc46fc72ab41812'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:30.544203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '850e54d515fd4233bbc46fc72ab41812'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:30.553280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:30.562232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '850e54d515fd4233bbc46fc72ab41812'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:30.572924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1014/2060] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:30 2022] DELETE /identity/v3/users/850e54d515fd4233bbc46fc72ab41812 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:30.575010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a135be7-7662-48c4-ae10-94d3585b4a34 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:30.575576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a135be7-7662-48c4-ae10-94d3585b4a34 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:30.575859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a135be7-7662-48c4-ae10-94d3585b4a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:30.594720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-6a135be7-7662-48c4-ae10-94d3585b4a34 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:30.595931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6a135be7-7662-48c4-ae10-94d3585b4a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:30.686238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1047/2061] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:30.695958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:30.699171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:30.815525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b4adc-2864-49a3-bded-adb2352b30da admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:30.820721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c1eef-a33e-4104-a06f-a5222d2e6b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:30.829420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1eef-a33e-4104-a06f-a5222d2e6b65 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:30.829420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1eef-a33e-4104-a06f-a5222d2e6b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:30.829420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1eef-a33e-4104-a06f-a5222d2e6b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:30.838811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c1eef-a33e-4104-a06f-a5222d2e6b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:30.841964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c1eef-a33e-4104-a06f-a5222d2e6b65 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:30.853835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1015/2062] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:31.730884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483e8d6-b282-4c58-a85e-de4cf2e0a3e2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:31.731758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483e8d6-b282-4c58-a85e-de4cf2e0a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:31.732043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483e8d6-b282-4c58-a85e-de4cf2e0a3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:31.762801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2483e8d6-b282-4c58-a85e-de4cf2e0a3e2 None None] MFA Rules not processed for user `bc76fd7a5bf24013a2d382657a29d4de`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:31.766153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2483e8d6-b282-4c58-a85e-de4cf2e0a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:31.769925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c1eef-a33e-4104-a06f-a5222d2e6b65 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:31.772453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ce1c1eef-a33e-4104-a06f-a5222d2e6b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:31.808708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c1eef-a33e-4104-a06f-a5222d2e6b65 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:31.818534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:31.818534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:31.819109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:31.819577 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] PATH_INFO: `/v3/projects/a89d390893f3469eb01534a788e9e1eb` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:31.821356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89d390893f3469eb01534a788e9e1eb)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:31.824001 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:31.900856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1048/2063] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:31.905393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483e8d6-b282-4c58-a85e-de4cf2e0a3e2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:31.906808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2483e8d6-b282-4c58-a85e-de4cf2e0a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:31.921215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483e8d6-b282-4c58-a85e-de4cf2e0a3e2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:31.926603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93397d81-9a64-4100-934a-e20891aa4555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:31.927945 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93397d81-9a64-4100-934a-e20891aa4555 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:31.931958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93397d81-9a64-4100-934a-e20891aa4555 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:31.931958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93397d81-9a64-4100-934a-e20891aa4555 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:31.931958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93397d81-9a64-4100-934a-e20891aa4555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:31.931958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93397d81-9a64-4100-934a-e20891aa4555 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:31.943241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89d390893f3469eb01534a788e9e1eb', 'request_id': 'req-0c510945-5f4b-46c5-8daa-756682775695'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:31.950554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89d390893f3469eb01534a788e9e1eb', 'request_id': 'req-0c510945-5f4b-46c5-8daa-756682775695'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:32.110287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89d390893f3469eb01534a788e9e1eb', 'request_id': 'req-0c510945-5f4b-46c5-8daa-756682775695'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:32.121582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1049/2064] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:31 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:32.127257 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93397d81-9a64-4100-934a-e20891aa4555 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:32.131907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] The token cache is being invalidate because project a89d390893f3469eb01534a788e9e1eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:32.132220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:32.135744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:32.150450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1016/2065] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:31 2022] DELETE /identity/v3/projects/a89d390893f3469eb01534a788e9e1eb => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:32.155043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:32.159720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:32.249959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:32.261453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93397d81-9a64-4100-934a-e20891aa4555 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:32.269383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50e623-55b9-402e-997e-09011816258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:32.271675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50e623-55b9-402e-997e-09011816258b admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:32.271675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50e623-55b9-402e-997e-09011816258b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:32.271675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50e623-55b9-402e-997e-09011816258b admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:32.271675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50e623-55b9-402e-997e-09011816258b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:32.275004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50e623-55b9-402e-997e-09011816258b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:32.280965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c510945-5f4b-46c5-8daa-756682775695 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:32.293610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:32.294600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:32.294961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:32.295892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] PATH_INFO: `/v3/users/5894cc21fe2b4c9eb4f6695f0e772d27` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:32.298951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5894cc21fe2b4c9eb4f6695f0e772d27)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:32.301863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:32.319315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:32.424815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1050/2066] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:32 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:32.428887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50e623-55b9-402e-997e-09011816258b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:32.431259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9f50e623-55b9-402e-997e-09011816258b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:32.444000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50e623-55b9-402e-997e-09011816258b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:32.449453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc8c79-5fb0-4a4f-8f6e-7629ffe91e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:32.450283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc8c79-5fb0-4a4f-8f6e-7629ffe91e83 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:32.451053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc8c79-5fb0-4a4f-8f6e-7629ffe91e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:32.451814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc8c79-5fb0-4a4f-8f6e-7629ffe91e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:32.452061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc8c79-5fb0-4a4f-8f6e-7629ffe91e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:32.453745 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc8c79-5fb0-4a4f-8f6e-7629ffe91e83 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:32.464721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1051/2067] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:32.469501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc8c79-5fb0-4a4f-8f6e-7629ffe91e83 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:32.479322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc8c79-5fb0-4a4f-8f6e-7629ffe91e83 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:32.485024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca1292-ce0e-4367-92b0-761133e5c38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:32.493290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca1292-ce0e-4367-92b0-761133e5c38f admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:32.493290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca1292-ce0e-4367-92b0-761133e5c38f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:32.493290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca1292-ce0e-4367-92b0-761133e5c38f admin admin] PATH_INFO: `/v3/projects/d7495b8c09f7452b9c50d9a2cf6048fb/users/797d8023bf5043d08706f3335a177601/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:32.501920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca1292-ce0e-4367-92b0-761133e5c38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7495b8c09f7452b9c50d9a2cf6048fb, user_id=797d8023bf5043d08706f3335a177601, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:32.504888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca1292-ce0e-4367-92b0-761133e5c38f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:32.554629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5894cc21fe2b4c9eb4f6695f0e772d27'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:32.561936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1052/2068] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:32 2022] PUT /identity/v3/projects/d7495b8c09f7452b9c50d9a2cf6048fb/users/797d8023bf5043d08706f3335a177601/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:32.565114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5894cc21fe2b4c9eb4f6695f0e772d27'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:32.565939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5894cc21fe2b4c9eb4f6695f0e772d27'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:32.566144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca1292-ce0e-4367-92b0-761133e5c38f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:32.643561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca1292-ce0e-4367-92b0-761133e5c38f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:32.649448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4f671-8ad5-4938-a382-6cb683c6600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:32.650135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4f671-8ad5-4938-a382-6cb683c6600b admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:32.650598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4f671-8ad5-4938-a382-6cb683c6600b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:32.651023 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4f671-8ad5-4938-a382-6cb683c6600b admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:32.651778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4f671-8ad5-4938-a382-6cb683c6600b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:32.652030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5894cc21fe2b4c9eb4f6695f0e772d27'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:32.653456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4f671-8ad5-4938-a382-6cb683c6600b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:32.668898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1053/2069] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:02:32.669082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5894cc21fe2b4c9eb4f6695f0e772d27'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:32.670177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4f671-8ad5-4938-a382-6cb683c6600b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:32.672226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f0e4f671-8ad5-4938-a382-6cb683c6600b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:32.675430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1017/2070] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:32 2022] DELETE /identity/v3/users/5894cc21fe2b4c9eb4f6695f0e772d27 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:32.693574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4f671-8ad5-4938-a382-6cb683c6600b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:32.771979 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407647a-9e6f-4d03-a5f3-f686c1300d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:32.772984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407647a-9e6f-4d03-a5f3-f686c1300d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:32.773255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407647a-9e6f-4d03-a5f3-f686c1300d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:32.773561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407647a-9e6f-4d03-a5f3-f686c1300d27 admin admin] PATH_INFO: `/v3/projects/d7495b8c09f7452b9c50d9a2cf6048fb/users/797d8023bf5043d08706f3335a177601/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:32.775713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407647a-9e6f-4d03-a5f3-f686c1300d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7495b8c09f7452b9c50d9a2cf6048fb, user_id=797d8023bf5043d08706f3335a177601, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:32.778449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407647a-9e6f-4d03-a5f3-f686c1300d27 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:32.789905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1054/2071] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:32 2022] PUT /identity/v3/projects/d7495b8c09f7452b9c50d9a2cf6048fb/users/797d8023bf5043d08706f3335a177601/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:33.216575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:33.218168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:33.246667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f33288-c769-489a-990f-6b7ec146a74f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:33.330108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:33.330849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:33.331226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:33.331598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] PATH_INFO: `/v3/projects/111b344df3f84d0591531cc33acefdfd` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:33.333038 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=111b344df3f84d0591531cc33acefdfd)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:33.335488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:33.478243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111b344df3f84d0591531cc33acefdfd', 'request_id': 'req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:33.492062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111b344df3f84d0591531cc33acefdfd', 'request_id': 'req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:33.605927 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111b344df3f84d0591531cc33acefdfd', 'request_id': 'req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:33.636887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] The token cache is being invalidate because project 111b344df3f84d0591531cc33acefdfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:33.637091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:33.637695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d70bf938-38a5-40c8-a08c-4c2ff53e7c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:33.640123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1018/2072] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:33 2022] DELETE /identity/v3/projects/111b344df3f84d0591531cc33acefdfd => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:33.645743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407647a-9e6f-4d03-a5f3-f686c1300d27 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:33.649973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5407647a-9e6f-4d03-a5f3-f686c1300d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:33.671661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407647a-9e6f-4d03-a5f3-f686c1300d27 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:33.755626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:33.778208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:33.779155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:33.779439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:33.779715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] PATH_INFO: `/v3/users/99f58bc3d7d64e2b87e34922b7416e55` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:33.781142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f58bc3d7d64e2b87e34922b7416e55)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:33.783121 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:33.952402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f58bc3d7d64e2b87e34922b7416e55'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:33.965591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f58bc3d7d64e2b87e34922b7416e55'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:33.972874 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f58bc3d7d64e2b87e34922b7416e55'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:33.973636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f58bc3d7d64e2b87e34922b7416e55'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.023021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b95eda7-7b8f-4d0c-a6ab-9a8169a8e376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f58bc3d7d64e2b87e34922b7416e55'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.030039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1055/2073] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:33 2022] DELETE /identity/v3/users/99f58bc3d7d64e2b87e34922b7416e55 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:34.118565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db55b94-da82-478e-9099-1d43a3563455 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:34.118863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db55b94-da82-478e-9099-1d43a3563455 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:34.119149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db55b94-da82-478e-9099-1d43a3563455 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:34.120731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1019/2074] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:02:34.141220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ded897-4dd3-49a5-9503-1c528b276dd5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:34.141640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ded897-4dd3-49a5-9503-1c528b276dd5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:34.142726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ded897-4dd3-49a5-9503-1c528b276dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:34.178768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-83ded897-4dd3-49a5-9503-1c528b276dd5 None None] MFA Rules not processed for user `1c1b23b83544423abc6919d805916b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:34.180566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-83ded897-4dd3-49a5-9503-1c528b276dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:34.271372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1056/2075] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:34.317052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db55b94-da82-478e-9099-1d43a3563455 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:34.324752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6db55b94-da82-478e-9099-1d43a3563455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:34.337012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ded897-4dd3-49a5-9503-1c528b276dd5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:34.337012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-83ded897-4dd3-49a5-9503-1c528b276dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:34.357582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db55b94-da82-478e-9099-1d43a3563455 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:34.369709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c5e0-6bdf-4b8f-a5bb-eee736183a0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:34.371170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c5e0-6bdf-4b8f-a5bb-eee736183a0c service placement] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:34.371458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c5e0-6bdf-4b8f-a5bb-eee736183a0c service placement] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:34.371738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c5e0-6bdf-4b8f-a5bb-eee736183a0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:34.389845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c5e0-6bdf-4b8f-a5bb-eee736183a0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:34.392372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c5e0-6bdf-4b8f-a5bb-eee736183a0c service placement] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:34.407279 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1020/2076] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:34.432089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:34.436935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ded897-4dd3-49a5-9503-1c528b276dd5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:34.442310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:34.443495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:34.443495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:34.443667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] PATH_INFO: `/v3/users/0602248c62664dd4a8809225d14ca5d7` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:34.446162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0602248c62664dd4a8809225d14ca5d7)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:34.452019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:34.553999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c5e0-6bdf-4b8f-a5bb-eee736183a0c service placement] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:34.556297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8199c5e0-6bdf-4b8f-a5bb-eee736183a0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:34.573584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c5e0-6bdf-4b8f-a5bb-eee736183a0c service placement] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:34.581216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:34.581216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:34.581589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:34.581717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] PATH_INFO: `/v3/projects/197522cec0a641d99c94df0db0654d5b` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:34.583261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197522cec0a641d99c94df0db0654d5b)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:34.585414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:34.595566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:34.645264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0602248c62664dd4a8809225d14ca5d7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.657973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0602248c62664dd4a8809225d14ca5d7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.665955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0602248c62664dd4a8809225d14ca5d7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.669429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0602248c62664dd4a8809225d14ca5d7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.756878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197522cec0a641d99c94df0db0654d5b', 'request_id': 'req-653c4de8-4607-4d44-b4d7-1e7ba8644398'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.757465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f797569a-02dc-4314-a13a-a913e1c632a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0602248c62664dd4a8809225d14ca5d7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.764484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197522cec0a641d99c94df0db0654d5b', 'request_id': 'req-653c4de8-4607-4d44-b4d7-1e7ba8644398'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.767754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1057/2077] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:34 2022] DELETE /identity/v3/users/0602248c62664dd4a8809225d14ca5d7 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:34.842795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197522cec0a641d99c94df0db0654d5b', 'request_id': 'req-653c4de8-4607-4d44-b4d7-1e7ba8644398'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.863165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] The token cache is being invalidate because project 197522cec0a641d99c94df0db0654d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:34.863474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.865058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-653c4de8-4607-4d44-b4d7-1e7ba8644398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:34.867504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1021/2078] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:34 2022] DELETE /identity/v3/projects/197522cec0a641d99c94df0db0654d5b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:34.874694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf5ddf-694e-40e2-851d-f60d2d003acb None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:34.875223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf5ddf-694e-40e2-851d-f60d2d003acb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:34.875300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf5ddf-694e-40e2-851d-f60d2d003acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:34.895133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c9cf5ddf-694e-40e2-851d-f60d2d003acb None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:34.896636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c9cf5ddf-694e-40e2-851d-f60d2d003acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:34.958912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1369ca-9827-4f34-bad2-4030e21479c2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:34.959194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1369ca-9827-4f34-bad2-4030e21479c2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:34.959480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1369ca-9827-4f34-bad2-4030e21479c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:34.997311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1058/2079] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:34.997543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-1a1369ca-9827-4f34-bad2-4030e21479c2 None None] MFA Rules not processed for user `797d8023bf5043d08706f3335a177601`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:34.997543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1a1369ca-9827-4f34-bad2-4030e21479c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.008880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38877812-c18a-4288-9a31-2ba47b2a4fb9 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.008880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38877812-c18a-4288-9a31-2ba47b2a4fb9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.008880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38877812-c18a-4288-9a31-2ba47b2a4fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.048880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-38877812-c18a-4288-9a31-2ba47b2a4fb9 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:35.055719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-38877812-c18a-4288-9a31-2ba47b2a4fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.065953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1059/2080] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:35.070338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38877812-c18a-4288-9a31-2ba47b2a4fb9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.083164 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38877812-c18a-4288-9a31-2ba47b2a4fb9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.084351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1022/2081] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:35.096492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d840f-a5ef-4aaf-88c4-c5ff4f1e89b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.096733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1369ca-9827-4f34-bad2-4030e21479c2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.097288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d840f-a5ef-4aaf-88c4-c5ff4f1e89b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.097583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d840f-a5ef-4aaf-88c4-c5ff4f1e89b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.097871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d840f-a5ef-4aaf-88c4-c5ff4f1e89b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.097957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1a1369ca-9827-4f34-bad2-4030e21479c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.098617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d840f-a5ef-4aaf-88c4-c5ff4f1e89b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:35.098939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d840f-a5ef-4aaf-88c4-c5ff4f1e89b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.100580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d840f-a5ef-4aaf-88c4-c5ff4f1e89b4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.116206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1060/2082] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:02:35.117992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d840f-a5ef-4aaf-88c4-c5ff4f1e89b4 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.128943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d840f-a5ef-4aaf-88c4-c5ff4f1e89b4 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.135262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9235f6e-367f-44f3-9dc6-f5e902daf8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.135965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9235f6e-367f-44f3-9dc6-f5e902daf8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.137301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9235f6e-367f-44f3-9dc6-f5e902daf8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.137301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9235f6e-367f-44f3-9dc6-f5e902daf8d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.137496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9235f6e-367f-44f3-9dc6-f5e902daf8d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.139278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9235f6e-367f-44f3-9dc6-f5e902daf8d4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.156630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1369ca-9827-4f34-bad2-4030e21479c2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.160389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.161092 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.161379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.161740 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.164575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.225777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.227252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.236572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1023/2083] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:35.237406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.238688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.252500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86293a-7b18-4503-b3b6-7228eaec1e36 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.257519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.258975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.259203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.259501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] PATH_INFO: `/v3/projects/c1d92a82166d4e128dbef71394daba86` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.260745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d92a82166d4e128dbef71394daba86)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.262758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.278669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1061/2084] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:35 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:35.282634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9235f6e-367f-44f3-9dc6-f5e902daf8d4 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.319653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:35.349305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9235f6e-367f-44f3-9dc6-f5e902daf8d4 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.354584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3501ec-70b0-4c43-98ff-49f4b2f5ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.355487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3501ec-70b0-4c43-98ff-49f4b2f5ac32 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.355851 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3501ec-70b0-4c43-98ff-49f4b2f5ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.356203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3501ec-70b0-4c43-98ff-49f4b2f5ac32 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.357011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3501ec-70b0-4c43-98ff-49f4b2f5ac32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.359092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3501ec-70b0-4c43-98ff-49f4b2f5ac32 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.392337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d92a82166d4e128dbef71394daba86', 'request_id': 'req-293d2de5-843e-47f6-aab3-1783fb74e22c'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:35.398847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d92a82166d4e128dbef71394daba86', 'request_id': 'req-293d2de5-843e-47f6-aab3-1783fb74e22c'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:35.461916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1062/2085] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:35 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:35.463265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3501ec-70b0-4c43-98ff-49f4b2f5ac32 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.474212 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3501ec-70b0-4c43-98ff-49f4b2f5ac32 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.497455 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d92a82166d4e128dbef71394daba86', 'request_id': 'req-293d2de5-843e-47f6-aab3-1783fb74e22c'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:35.510953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] The token cache is being invalidate because project c1d92a82166d4e128dbef71394daba86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:35.511172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:35.511849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-293d2de5-843e-47f6-aab3-1783fb74e22c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:35.513755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1024/2086] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:35 2022] DELETE /identity/v3/projects/c1d92a82166d4e128dbef71394daba86 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:35.519822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926bf9c-a9fb-440f-8a94-a778d94ccbe2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.520304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926bf9c-a9fb-440f-8a94-a778d94ccbe2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.520594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926bf9c-a9fb-440f-8a94-a778d94ccbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.546946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c926bf9c-a9fb-440f-8a94-a778d94ccbe2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:35.546946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c926bf9c-a9fb-440f-8a94-a778d94ccbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.547272 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349e461-ffaa-450f-900f-ea4f7b56f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.547992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349e461-ffaa-450f-900f-ea4f7b56f088 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.548286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349e461-ffaa-450f-900f-ea4f7b56f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.548571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349e461-ffaa-450f-900f-ea4f7b56f088 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.549420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349e461-ffaa-450f-900f-ea4f7b56f088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.550994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349e461-ffaa-450f-900f-ea4f7b56f088 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.556506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1025/2087] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:35.563165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1063/2088] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:02:35.565550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554d521-4cf3-4790-833d-3826dacf8761 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.565855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554d521-4cf3-4790-833d-3826dacf8761 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.569805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554d521-4cf3-4790-833d-3826dacf8761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.573108 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349e461-ffaa-450f-900f-ea4f7b56f088 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.574826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7349e461-ffaa-450f-900f-ea4f7b56f088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.586689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-9554d521-4cf3-4790-833d-3826dacf8761 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:35.587202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349e461-ffaa-450f-900f-ea4f7b56f088 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.587900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9554d521-4cf3-4790-833d-3826dacf8761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.592068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9eef65-061d-48a4-8651-d0048dcb0e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.592773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9eef65-061d-48a4-8651-d0048dcb0e32 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.593061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9eef65-061d-48a4-8651-d0048dcb0e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.593337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9eef65-061d-48a4-8651-d0048dcb0e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.594119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9eef65-061d-48a4-8651-d0048dcb0e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.595868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9eef65-061d-48a4-8651-d0048dcb0e32 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.596747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1026/2089] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:35.601248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554d521-4cf3-4790-833d-3826dacf8761 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.607152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1064/2090] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:02:35.613096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9eef65-061d-48a4-8651-d0048dcb0e32 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.614047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554d521-4cf3-4790-833d-3826dacf8761 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.621347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94232b-c74e-4a6c-838a-a51f4079dd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.622232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94232b-c74e-4a6c-838a-a51f4079dd36 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.622525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94232b-c74e-4a6c-838a-a51f4079dd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.622806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94232b-c74e-4a6c-838a-a51f4079dd36 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.623630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94232b-c74e-4a6c-838a-a51f4079dd36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:35.624095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94232b-c74e-4a6c-838a-a51f4079dd36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.625662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94232b-c74e-4a6c-838a-a51f4079dd36 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.629032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9eef65-061d-48a4-8651-d0048dcb0e32 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.630453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:35.634528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3f904-8978-4208-b86e-35b1ae5f4482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.635541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3f904-8978-4208-b86e-35b1ae5f4482 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.635834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3f904-8978-4208-b86e-35b1ae5f4482 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.636112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3f904-8978-4208-b86e-35b1ae5f4482 admin admin] PATH_INFO: `/v3/projects/126709ddf9e6431cbb28317d0bd4de27/users/51a44ebaf5a64f738cc6ad3d326b2ffb/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.640546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1027/2091] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:02:35.644933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94232b-c74e-4a6c-838a-a51f4079dd36 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.651657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3f904-8978-4208-b86e-35b1ae5f4482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=126709ddf9e6431cbb28317d0bd4de27, user_id=51a44ebaf5a64f738cc6ad3d326b2ffb, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.654399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3f904-8978-4208-b86e-35b1ae5f4482 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.654706 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94232b-c74e-4a6c-838a-a51f4079dd36 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.659534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0226a-2b8f-42b5-a696-1d1f3e0c2b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.660195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0226a-2b8f-42b5-a696-1d1f3e0c2b36 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.660484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0226a-2b8f-42b5-a696-1d1f3e0c2b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.660800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0226a-2b8f-42b5-a696-1d1f3e0c2b36 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.661496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0226a-2b8f-42b5-a696-1d1f3e0c2b36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.663414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0226a-2b8f-42b5-a696-1d1f3e0c2b36 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.737184 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1065/2092] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:35 2022] PUT /identity/v3/projects/126709ddf9e6431cbb28317d0bd4de27/users/51a44ebaf5a64f738cc6ad3d326b2ffb/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:35.748537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3f904-8978-4208-b86e-35b1ae5f4482 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.750597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-88c3f904-8978-4208-b86e-35b1ae5f4482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.784878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1028/2093] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:35 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:35.786058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0226a-2b8f-42b5-a696-1d1f3e0c2b36 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:35.790458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-efd0226a-2b8f-42b5-a696-1d1f3e0c2b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:35.829891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3f904-8978-4208-b86e-35b1ae5f4482 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.893948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0226a-2b8f-42b5-a696-1d1f3e0c2b36 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:35.896246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.897340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.897627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.897908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] PATH_INFO: `/v3/users/1ffc8087dadc4290b7b9268cef1a4984` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.899176 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ffc8087dadc4290b7b9268cef1a4984)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.901263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:35.901385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:35.902681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:35.903128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:35.903471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] PATH_INFO: `/v3/users/9f2eb949f1a24163833fb07b01215555` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:35.905010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f2eb949f1a24163833fb07b01215555)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:35.907006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.224928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f2eb949f1a24163833fb07b01215555'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.233053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f2eb949f1a24163833fb07b01215555'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.233783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f2eb949f1a24163833fb07b01215555'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.236949 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ffc8087dadc4290b7b9268cef1a4984'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.249215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ffc8087dadc4290b7b9268cef1a4984'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.255993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ffc8087dadc4290b7b9268cef1a4984'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.256742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ffc8087dadc4290b7b9268cef1a4984'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.327662 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f2eb949f1a24163833fb07b01215555'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.342435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-631569b5-3ed1-427d-9ce8-b460c0c6181d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f2eb949f1a24163833fb07b01215555'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.348737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1029/2094] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:35 2022] DELETE /identity/v3/users/9f2eb949f1a24163833fb07b01215555 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:36.350127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc3bea-85a1-4b90-a91f-ce9ed57cbae6 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.350415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc3bea-85a1-4b90-a91f-ce9ed57cbae6 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.350696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc3bea-85a1-4b90-a91f-ce9ed57cbae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.378337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-34cc3bea-85a1-4b90-a91f-ce9ed57cbae6 None None] MFA Rules not processed for user `51a44ebaf5a64f738cc6ad3d326b2ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:36.379594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-34cc3bea-85a1-4b90-a91f-ce9ed57cbae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:36.418098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ffc8087dadc4290b7b9268cef1a4984'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:36.426664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1066/2095] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:35 2022] DELETE /identity/v3/users/1ffc8087dadc4290b7b9268cef1a4984 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:36.427225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:36.430162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:36.457514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1030/2096] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:36.460174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a9faf-164c-4fbb-a332-927b1455d2fe None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.460470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a9faf-164c-4fbb-a332-927b1455d2fe None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.460803 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a9faf-164c-4fbb-a332-927b1455d2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.479176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8f8a9faf-164c-4fbb-a332-927b1455d2fe None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:36.481002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8f8a9faf-164c-4fbb-a332-927b1455d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:36.501705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d14609-311f-406e-a7da-b5cecb075a8d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:36.506662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe4ece-839a-4a44-82ee-f63a30586816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:36.507713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe4ece-839a-4a44-82ee-f63a30586816 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.507713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe4ece-839a-4a44-82ee-f63a30586816 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.508625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe4ece-839a-4a44-82ee-f63a30586816 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.509548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe4ece-839a-4a44-82ee-f63a30586816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:36.511459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe4ece-839a-4a44-82ee-f63a30586816 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.545761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1031/2097] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:36.547086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d965f11-807e-46be-8696-2dcd46cd9fdb None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.547386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d965f11-807e-46be-8696-2dcd46cd9fdb None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.547670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d965f11-807e-46be-8696-2dcd46cd9fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.571515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-5d965f11-807e-46be-8696-2dcd46cd9fdb None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:36.572757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5d965f11-807e-46be-8696-2dcd46cd9fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:36.583824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1032/2098] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:36.583824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d965f11-807e-46be-8696-2dcd46cd9fdb None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:36.583824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5d965f11-807e-46be-8696-2dcd46cd9fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:36.594910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d965f11-807e-46be-8696-2dcd46cd9fdb None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:36.599564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78fd84-2589-4d87-8f47-1572b9fbb752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:36.600229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78fd84-2589-4d87-8f47-1572b9fbb752 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.600518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78fd84-2589-4d87-8f47-1572b9fbb752 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.600831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78fd84-2589-4d87-8f47-1572b9fbb752 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.601701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78fd84-2589-4d87-8f47-1572b9fbb752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:36.603610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78fd84-2589-4d87-8f47-1572b9fbb752 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.607212 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1067/2099] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:36 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:36.608225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe4ece-839a-4a44-82ee-f63a30586816 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:36.610580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-51fe4ece-839a-4a44-82ee-f63a30586816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:36.667980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe4ece-839a-4a44-82ee-f63a30586816 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:36.672098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65a99c-f825-4643-b22b-c24e32f817bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:36.672795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65a99c-f825-4643-b22b-c24e32f817bc service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.673084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65a99c-f825-4643-b22b-c24e32f817bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.673364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65a99c-f825-4643-b22b-c24e32f817bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.685243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65a99c-f825-4643-b22b-c24e32f817bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:36.686698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65a99c-f825-4643-b22b-c24e32f817bc service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.695389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1033/2100] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:36 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:36.696765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78fd84-2589-4d87-8f47-1572b9fbb752 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:36.698357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-bf78fd84-2589-4d87-8f47-1572b9fbb752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:36.701009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1068/2101] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:36.701868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65a99c-f825-4643-b22b-c24e32f817bc service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:36.712729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65a99c-f825-4643-b22b-c24e32f817bc service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:36.765276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78fd84-2589-4d87-8f47-1572b9fbb752 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:36.771125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7846c2a-bf31-4b9d-9ebb-2fa9dfdc1623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:36.774750 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7846c2a-bf31-4b9d-9ebb-2fa9dfdc1623 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.774750 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7846c2a-bf31-4b9d-9ebb-2fa9dfdc1623 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.774750 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7846c2a-bf31-4b9d-9ebb-2fa9dfdc1623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.786253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa8de9-869b-4ac5-a785-a3a83af38353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:36.787241 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa8de9-869b-4ac5-a785-a3a83af38353 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.787535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa8de9-869b-4ac5-a785-a3a83af38353 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.787850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa8de9-869b-4ac5-a785-a3a83af38353 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.788566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa8de9-869b-4ac5-a785-a3a83af38353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:36.790262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa8de9-869b-4ac5-a785-a3a83af38353 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.803186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1069/2102] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:36.807036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa8de9-869b-4ac5-a785-a3a83af38353 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:36.817436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa8de9-869b-4ac5-a785-a3a83af38353 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:36.825435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:36.829044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cefcc-ffb9-4212-911b-a93dbbecd1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:36.832758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cefcc-ffb9-4212-911b-a93dbbecd1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.832758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cefcc-ffb9-4212-911b-a93dbbecd1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.832758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cefcc-ffb9-4212-911b-a93dbbecd1e8 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.832758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cefcc-ffb9-4212-911b-a93dbbecd1e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:36.833821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cefcc-ffb9-4212-911b-a93dbbecd1e8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.848826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7846c2a-bf31-4b9d-9ebb-2fa9dfdc1623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:36.850232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7846c2a-bf31-4b9d-9ebb-2fa9dfdc1623 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.860055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1034/2103] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:36.860921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7846c2a-bf31-4b9d-9ebb-2fa9dfdc1623 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:36.870923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7846c2a-bf31-4b9d-9ebb-2fa9dfdc1623 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:36.875700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99841fb0-02c4-4fde-b9cd-e25d0a4effcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:36.876385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99841fb0-02c4-4fde-b9cd-e25d0a4effcc admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.876699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99841fb0-02c4-4fde-b9cd-e25d0a4effcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.876981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99841fb0-02c4-4fde-b9cd-e25d0a4effcc admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.878575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99841fb0-02c4-4fde-b9cd-e25d0a4effcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:36.879357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99841fb0-02c4-4fde-b9cd-e25d0a4effcc admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.891542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1035/2104] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:36.895987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99841fb0-02c4-4fde-b9cd-e25d0a4effcc admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:36.906248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99841fb0-02c4-4fde-b9cd-e25d0a4effcc admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:36.911536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef1d9a-d7b2-40c9-8eb6-2573431dcca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:36.912449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef1d9a-d7b2-40c9-8eb6-2573431dcca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.912770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef1d9a-d7b2-40c9-8eb6-2573431dcca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.913054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef1d9a-d7b2-40c9-8eb6-2573431dcca7 admin admin] PATH_INFO: `/v3/projects/dcc8f2e8d4e64866a624dcd6d1de5d81/users/822e56f086f0481ca4c6970adf85e396/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.947913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef1d9a-d7b2-40c9-8eb6-2573431dcca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc8f2e8d4e64866a624dcd6d1de5d81, user_id=822e56f086f0481ca4c6970adf85e396, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:36.948682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1070/2105] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:36 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:36.951768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cefcc-ffb9-4212-911b-a93dbbecd1e8 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:36.954693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef1d9a-d7b2-40c9-8eb6-2573431dcca7 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.978265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cefcc-ffb9-4212-911b-a93dbbecd1e8 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:36.980589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa5a88-d4da-4de2-b48f-15eba7452611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:36.981305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa5a88-d4da-4de2-b48f-15eba7452611 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:36.981596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa5a88-d4da-4de2-b48f-15eba7452611 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:36.981909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa5a88-d4da-4de2-b48f-15eba7452611 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:36.982756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa5a88-d4da-4de2-b48f-15eba7452611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:36.984229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa5a88-d4da-4de2-b48f-15eba7452611 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:36.999589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1071/2106] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:37.010429 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa5a88-d4da-4de2-b48f-15eba7452611 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:37.023888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa5a88-d4da-4de2-b48f-15eba7452611 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:37.039322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1036/2107] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:36 2022] PUT /identity/v3/projects/dcc8f2e8d4e64866a624dcd6d1de5d81/users/822e56f086f0481ca4c6970adf85e396/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:37.041164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e65da-2ade-4d93-8c4e-c8313b593692 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:37.041529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e65da-2ade-4d93-8c4e-c8313b593692 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:37.041881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e65da-2ade-4d93-8c4e-c8313b593692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:37.065385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-552e65da-2ade-4d93-8c4e-c8313b593692 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:37.066748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-552e65da-2ade-4d93-8c4e-c8313b593692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:37.139141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:37.194049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524de8a-b48f-4d4c-ae8b-217380aab881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:37.195117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524de8a-b48f-4d4c-ae8b-217380aab881 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:37.195621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524de8a-b48f-4d4c-ae8b-217380aab881 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:37.196519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524de8a-b48f-4d4c-ae8b-217380aab881 admin admin] PATH_INFO: `/v3/projects/e4a804a7fd4645c8ae05d67a75cbc8e3/users/265f2c7eaf5a408f9ed96db672d3d2e8/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:37.218261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1037/2108] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:37.219387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e65da-2ade-4d93-8c4e-c8313b593692 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:37.230534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524de8a-b48f-4d4c-ae8b-217380aab881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a804a7fd4645c8ae05d67a75cbc8e3, user_id=265f2c7eaf5a408f9ed96db672d3d2e8, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:37.231623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-552e65da-2ade-4d93-8c4e-c8313b593692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:37.238463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524de8a-b48f-4d4c-ae8b-217380aab881 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:37.337307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e65da-2ade-4d93-8c4e-c8313b593692 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:37.344173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:37.345303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:37.345731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:37.347085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:37.350201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:37.393075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1072/2109] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:37 2022] PUT /identity/v3/projects/e4a804a7fd4645c8ae05d67a75cbc8e3/users/265f2c7eaf5a408f9ed96db672d3d2e8/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:37.393800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524de8a-b48f-4d4c-ae8b-217380aab881 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:37.468529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:37.468529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:37.480545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1038/2110] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:37.481754 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:37.483569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:37.504989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcfd38-e65f-4236-aef1-b1599b0aa5e0 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:37.507293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524de8a-b48f-4d4c-ae8b-217380aab881 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:37.513153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3ad0-76d1-45b0-9dff-70cef933f689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:37.519512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3ad0-76d1-45b0-9dff-70cef933f689 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:37.521365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3ad0-76d1-45b0-9dff-70cef933f689 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:37.526004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3ad0-76d1-45b0-9dff-70cef933f689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:37.526004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:37.591289 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:37.594964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:37.594964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:37.594964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] PATH_INFO: `/v3/projects/d4ff6904678f4a49a91913587c5b6e3f` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:37.594964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4ff6904678f4a49a91913587c5b6e3f)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:37.597232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:37.647578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b3ad0-76d1-45b0-9dff-70cef933f689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:37.649641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b3ad0-76d1-45b0-9dff-70cef933f689 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:37.660999 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1073/2111] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:37.674469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3ad0-76d1-45b0-9dff-70cef933f689 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:37.681216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3ad0-76d1-45b0-9dff-70cef933f689 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:37.689107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4298c7-85d3-4d09-b833-f459bc7bd662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:37.689803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4298c7-85d3-4d09-b833-f459bc7bd662 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:37.690106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4298c7-85d3-4d09-b833-f459bc7bd662 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:37.690385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4298c7-85d3-4d09-b833-f459bc7bd662 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:37.691579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4298c7-85d3-4d09-b833-f459bc7bd662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:37.693345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4298c7-85d3-4d09-b833-f459bc7bd662 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:37.709829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1074/2112] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:37.709829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4298c7-85d3-4d09-b833-f459bc7bd662 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:37.709829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ed4298c7-85d3-4d09-b833-f459bc7bd662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:37.725181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4298c7-85d3-4d09-b833-f459bc7bd662 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:37.733174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:37.733759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:37.734076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:37.734338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] PATH_INFO: `/v3/projects/c3534114306a4599983b3d6389369ef3` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:37.735638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3534114306a4599983b3d6389369ef3)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:37.738001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:37.750777 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4ff6904678f4a49a91913587c5b6e3f', 'request_id': 'req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.760876 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4ff6904678f4a49a91913587c5b6e3f', 'request_id': 'req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.842489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4ff6904678f4a49a91913587c5b6e3f', 'request_id': 'req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.858816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] The token cache is being invalidate because project d4ff6904678f4a49a91913587c5b6e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:37.859031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.859701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.861348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1039/2113] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:37 2022] DELETE /identity/v3/projects/d4ff6904678f4a49a91913587c5b6e3f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:37.862108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:37.863440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:37.869181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3534114306a4599983b3d6389369ef3', 'request_id': 'req-4e48abcd-9630-49ae-8dee-d045069ce4cf'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.877413 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3534114306a4599983b3d6389369ef3', 'request_id': 'req-4e48abcd-9630-49ae-8dee-d045069ce4cf'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.885528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3534114306a4599983b3d6389369ef3', 'request_id': 'req-4e48abcd-9630-49ae-8dee-d045069ce4cf'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.938577 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdd0ca-a764-4276-bb72-9259c0d6c8c6 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:37.963835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] The token cache is being invalidate because project c3534114306a4599983b3d6389369ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:37.963835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.964994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4e48abcd-9630-49ae-8dee-d045069ce4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:37.967639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1075/2114] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:37 2022] DELETE /identity/v3/projects/c3534114306a4599983b3d6389369ef3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:37.969534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed455584-28ae-4dfd-89a1-fc4fd4600758 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:37.970091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed455584-28ae-4dfd-89a1-fc4fd4600758 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:37.970493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed455584-28ae-4dfd-89a1-fc4fd4600758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:37.986064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:37.997226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ed455584-28ae-4dfd-89a1-fc4fd4600758 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:37.998751 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ed455584-28ae-4dfd-89a1-fc4fd4600758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:38.020894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ba2a7-da5d-469f-b1b8-97080c1691de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:38.021866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ba2a7-da5d-469f-b1b8-97080c1691de admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.022202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ba2a7-da5d-469f-b1b8-97080c1691de admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.022692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ba2a7-da5d-469f-b1b8-97080c1691de admin admin] PATH_INFO: `/v3/projects/e4a804a7fd4645c8ae05d67a75cbc8e3/users/265f2c7eaf5a408f9ed96db672d3d2e8/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.027521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ba2a7-da5d-469f-b1b8-97080c1691de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a804a7fd4645c8ae05d67a75cbc8e3, user_id=265f2c7eaf5a408f9ed96db672d3d2e8, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:38.031759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ba2a7-da5d-469f-b1b8-97080c1691de admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:38.044226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1040/2115] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:37 2022] PUT /identity/v3/projects/e4a804a7fd4645c8ae05d67a75cbc8e3/users/265f2c7eaf5a408f9ed96db672d3d2e8/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:38.059144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ba2a7-da5d-469f-b1b8-97080c1691de admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:38.059144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e39ba2a7-da5d-469f-b1b8-97080c1691de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:38.107626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1076/2116] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:38.128563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4aa93-26b6-4c9e-9e29-10b5835a513d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.128563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4aa93-26b6-4c9e-9e29-10b5835a513d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.128563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4aa93-26b6-4c9e-9e29-10b5835a513d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.132809 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ba2a7-da5d-469f-b1b8-97080c1691de admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:38.138832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:38.139727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.140132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.140392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] PATH_INFO: `/v3/users/0d82fabd3e444471a6d2aabd7b2fa20a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.141067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-78c4aa93-26b6-4c9e-9e29-10b5835a513d None None] MFA Rules not processed for user `265f2c7eaf5a408f9ed96db672d3d2e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:38.141799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d82fabd3e444471a6d2aabd7b2fa20a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:38.142243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-78c4aa93-26b6-4c9e-9e29-10b5835a513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:38.143749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:38.217608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1077/2117] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:38.218998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb7bd6-6b3b-4cc5-9fef-9c0e740899d6 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.219293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb7bd6-6b3b-4cc5-9fef-9c0e740899d6 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.219570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb7bd6-6b3b-4cc5-9fef-9c0e740899d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.240193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-b7cb7bd6-6b3b-4cc5-9fef-9c0e740899d6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:38.241430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b7cb7bd6-6b3b-4cc5-9fef-9c0e740899d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:38.252717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1078/2118] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:38.252717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb7bd6-6b3b-4cc5-9fef-9c0e740899d6 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:38.253957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b7cb7bd6-6b3b-4cc5-9fef-9c0e740899d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:38.315089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb7bd6-6b3b-4cc5-9fef-9c0e740899d6 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:38.319006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ec724-65ba-446c-846a-660a9d9488fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:38.319668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ec724-65ba-446c-846a-660a9d9488fc service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.319950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ec724-65ba-446c-846a-660a9d9488fc service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.320267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ec724-65ba-446c-846a-660a9d9488fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.330466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ec724-65ba-446c-846a-660a9d9488fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:38.331915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ec724-65ba-446c-846a-660a9d9488fc service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:38.347377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1079/2119] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:38.347377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ec724-65ba-446c-846a-660a9d9488fc service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:38.358489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ec724-65ba-446c-846a-660a9d9488fc service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:38.365686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14524c68-97c2-42b6-8778-ded360d2d9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:38.366521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14524c68-97c2-42b6-8778-ded360d2d9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.367271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14524c68-97c2-42b6-8778-ded360d2d9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.367271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14524c68-97c2-42b6-8778-ded360d2d9b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.368736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14524c68-97c2-42b6-8778-ded360d2d9b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:38.369121 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14524c68-97c2-42b6-8778-ded360d2d9b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:38.370946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14524c68-97c2-42b6-8778-ded360d2d9b1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:38.386938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1080/2120] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:02:38.391386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14524c68-97c2-42b6-8778-ded360d2d9b1 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:38.401914 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14524c68-97c2-42b6-8778-ded360d2d9b1 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:38.406846 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df90d0-c144-4ddc-b33a-3e29a01e261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:38.407571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df90d0-c144-4ddc-b33a-3e29a01e261b admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.408074 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df90d0-c144-4ddc-b33a-3e29a01e261b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.408373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df90d0-c144-4ddc-b33a-3e29a01e261b admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.409152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df90d0-c144-4ddc-b33a-3e29a01e261b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:38.411119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df90d0-c144-4ddc-b33a-3e29a01e261b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:38.463729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d82fabd3e444471a6d2aabd7b2fa20a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:38.471655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d82fabd3e444471a6d2aabd7b2fa20a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:38.472419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d82fabd3e444471a6d2aabd7b2fa20a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:38.541557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1081/2121] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:38 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:38.546777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df90d0-c144-4ddc-b33a-3e29a01e261b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:38.561715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d82fabd3e444471a6d2aabd7b2fa20a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:38.567780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:38.582526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d82fabd3e444471a6d2aabd7b2fa20a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:38.591588 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1041/2122] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:38 2022] DELETE /identity/v3/users/0d82fabd3e444471a6d2aabd7b2fa20a => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:38.644572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df90d0-c144-4ddc-b33a-3e29a01e261b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:38.700248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:38.701998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:38.769792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807fe70-3032-44db-baac-3d05369e9220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:38.775704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807fe70-3032-44db-baac-3d05369e9220 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.775704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807fe70-3032-44db-baac-3d05369e9220 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.775704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807fe70-3032-44db-baac-3d05369e9220 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.775704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807fe70-3032-44db-baac-3d05369e9220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:38.775704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807fe70-3032-44db-baac-3d05369e9220 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:38.792278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b69a05-3d3d-4eba-93e8-700517533972 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:38.800975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e9428-dd5f-4ce2-b448-495782b2afd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:38.804415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e9428-dd5f-4ce2-b448-495782b2afd2 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.804415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e9428-dd5f-4ce2-b448-495782b2afd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.804415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e9428-dd5f-4ce2-b448-495782b2afd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.804415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d13e9428-dd5f-4ce2-b448-495782b2afd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:38.884671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1082/2123] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:38 2022] POST /identity/v3/users => generated 292 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:38.889567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807fe70-3032-44db-baac-3d05369e9220 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:38.894775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8807fe70-3032-44db-baac-3d05369e9220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:38.899959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e9428-dd5f-4ce2-b448-495782b2afd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:38.903042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e9428-dd5f-4ce2-b448-495782b2afd2 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:38.919748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1042/2124] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:38.934057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807fe70-3032-44db-baac-3d05369e9220 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:38.954785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d680b-9ff8-4920-9a83-04ff9b2d8da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:38.955553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d680b-9ff8-4920-9a83-04ff9b2d8da5 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:38.955844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d680b-9ff8-4920-9a83-04ff9b2d8da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:38.956124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d680b-9ff8-4920-9a83-04ff9b2d8da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:38.956964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d680b-9ff8-4920-9a83-04ff9b2d8da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:38.960216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d680b-9ff8-4920-9a83-04ff9b2d8da5 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:38.985339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1083/2125] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:38.994083 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e9428-dd5f-4ce2-b448-495782b2afd2 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:39.005500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e9428-dd5f-4ce2-b448-495782b2afd2 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:39.014452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2a347-8655-489d-b6b4-0b45a0868c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:39.015274 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2a347-8655-489d-b6b4-0b45a0868c01 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:39.015589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2a347-8655-489d-b6b4-0b45a0868c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:39.015873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2a347-8655-489d-b6b4-0b45a0868c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:39.016974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2a347-8655-489d-b6b4-0b45a0868c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:39.022989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2a347-8655-489d-b6b4-0b45a0868c01 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:39.045903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1043/2126] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:39.047842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d680b-9ff8-4920-9a83-04ff9b2d8da5 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:39.053669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:39.067221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d680b-9ff8-4920-9a83-04ff9b2d8da5 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:39.077244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c13858-ef93-4b2e-a7c8-16cc35549f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:39.077688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c13858-ef93-4b2e-a7c8-16cc35549f9b admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:39.078246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c13858-ef93-4b2e-a7c8-16cc35549f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:39.080857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c13858-ef93-4b2e-a7c8-16cc35549f9b admin admin] PATH_INFO: `/v3/projects/4d31f280cfc640dd8381ee0869d9f455/users/3f1b6145d67d4cbf9dcd398736e6b0da/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:39.109836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c13858-ef93-4b2e-a7c8-16cc35549f9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d31f280cfc640dd8381ee0869d9f455, user_id=3f1b6145d67d4cbf9dcd398736e6b0da, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:39.115828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c13858-ef93-4b2e-a7c8-16cc35549f9b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:39.180783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:39.253606 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1084/2127] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:39 2022] PUT /identity/v3/projects/4d31f280cfc640dd8381ee0869d9f455/users/3f1b6145d67d4cbf9dcd398736e6b0da/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:39.258786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3d3d6-4abc-469b-80d8-2f6f1a1ac8be None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:39.259085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3d3d6-4abc-469b-80d8-2f6f1a1ac8be None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:39.259363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3d3d6-4abc-469b-80d8-2f6f1a1ac8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:39.277654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d7d3d3d6-4abc-469b-80d8-2f6f1a1ac8be None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:39.278848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d7d3d3d6-4abc-469b-80d8-2f6f1a1ac8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:39.299041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c13858-ef93-4b2e-a7c8-16cc35549f9b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:39.303205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-30c13858-ef93-4b2e-a7c8-16cc35549f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:39.401335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c13858-ef93-4b2e-a7c8-16cc35549f9b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:39.406895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1044/2128] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:39.416243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:39.416243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:39.416243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:39.416243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] PATH_INFO: `/v3/projects/db727c61d76449118493687e55e231d4` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:39.417618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db727c61d76449118493687e55e231d4)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:39.420240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3d3d6-4abc-469b-80d8-2f6f1a1ac8be None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:39.421043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:39.422040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d7d3d3d6-4abc-469b-80d8-2f6f1a1ac8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:39.519406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3d3d6-4abc-469b-80d8-2f6f1a1ac8be None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:39.526373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a97b38-f002-4b77-955a-8782a2ddb71c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:39.530997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a97b38-f002-4b77-955a-8782a2ddb71c service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:39.530997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a97b38-f002-4b77-955a-8782a2ddb71c service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:39.530997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a97b38-f002-4b77-955a-8782a2ddb71c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:39.538578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a97b38-f002-4b77-955a-8782a2ddb71c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:39.540218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a97b38-f002-4b77-955a-8782a2ddb71c service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:39.553545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1045/2129] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:39.610760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db727c61d76449118493687e55e231d4', 'request_id': 'req-9eceb669-fe52-4897-8c33-4a15e8015b25'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:39.618512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db727c61d76449118493687e55e231d4', 'request_id': 'req-9eceb669-fe52-4897-8c33-4a15e8015b25'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:39.627416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db727c61d76449118493687e55e231d4', 'request_id': 'req-9eceb669-fe52-4897-8c33-4a15e8015b25'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:39.734635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] The token cache is being invalidate because project db727c61d76449118493687e55e231d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:39.734929 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:39.736512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9eceb669-fe52-4897-8c33-4a15e8015b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:39.737978 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1085/2130] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:39 2022] DELETE /identity/v3/projects/db727c61d76449118493687e55e231d4 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:39.744242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc22ec4-4fe5-4afb-86c9-ffa4291d18a6 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:39.744452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc22ec4-4fe5-4afb-86c9-ffa4291d18a6 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:39.744792 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc22ec4-4fe5-4afb-86c9-ffa4291d18a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:39.775825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-4bc22ec4-4fe5-4afb-86c9-ffa4291d18a6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:39.785449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4bc22ec4-4fe5-4afb-86c9-ffa4291d18a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:39.874475 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1046/2131] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:39.884399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77047047-c929-49bd-bdf5-c02137e061ec None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:39.884808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77047047-c929-49bd-bdf5-c02137e061ec None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:39.885148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77047047-c929-49bd-bdf5-c02137e061ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:39.924252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-77047047-c929-49bd-bdf5-c02137e061ec None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:39.925756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-77047047-c929-49bd-bdf5-c02137e061ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:39.943800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1086/2132] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:39.948414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc22ec4-4fe5-4afb-86c9-ffa4291d18a6 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:39.966677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc22ec4-4fe5-4afb-86c9-ffa4291d18a6 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:39.973499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c519e-f724-4260-bd41-fffadf63b396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:39.974817 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c519e-f724-4260-bd41-fffadf63b396 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:39.975202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c519e-f724-4260-bd41-fffadf63b396 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:39.976532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c519e-f724-4260-bd41-fffadf63b396 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:39.977889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c519e-f724-4260-bd41-fffadf63b396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:39.978310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c519e-f724-4260-bd41-fffadf63b396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:39.980938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c519e-f724-4260-bd41-fffadf63b396 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:39.998182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1047/2133] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:02:40.002340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77047047-c929-49bd-bdf5-c02137e061ec None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:40.012884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77047047-c929-49bd-bdf5-c02137e061ec None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:40.038678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac582ef-21d0-44ac-9dd1-9a7bac766cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:40.040991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac582ef-21d0-44ac-9dd1-9a7bac766cef admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:40.042677 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac582ef-21d0-44ac-9dd1-9a7bac766cef admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:40.044472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac582ef-21d0-44ac-9dd1-9a7bac766cef admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:40.053332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac582ef-21d0-44ac-9dd1-9a7bac766cef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:40.057269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac582ef-21d0-44ac-9dd1-9a7bac766cef admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:40.204764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1087/2134] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:40 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:40.207316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c519e-f724-4260-bd41-fffadf63b396 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:40.232412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c519e-f724-4260-bd41-fffadf63b396 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:40.259553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac582ef-21d0-44ac-9dd1-9a7bac766cef admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:40.261558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0ac582ef-21d0-44ac-9dd1-9a7bac766cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:40.329070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:40.346095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73b7b8-4310-445a-a547-11f45ba62fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:40.346808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73b7b8-4310-445a-a547-11f45ba62fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:40.347097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73b7b8-4310-445a-a547-11f45ba62fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:40.347376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73b7b8-4310-445a-a547-11f45ba62fe0 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:40.348098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73b7b8-4310-445a-a547-11f45ba62fe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:40.358851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73b7b8-4310-445a-a547-11f45ba62fe0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:40.446651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1048/2135] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:40 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:02:40.448370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73b7b8-4310-445a-a547-11f45ba62fe0 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:40.451654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac582ef-21d0-44ac-9dd1-9a7bac766cef admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:40.459258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:40.460159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:40.460450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73b7b8-4310-445a-a547-11f45ba62fe0 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:40.460579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:40.460751 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] PATH_INFO: `/v3/users/b723d57a7dc44b4d8f8af2a9f47c614f` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:40.465828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b723d57a7dc44b4d8f8af2a9f47c614f)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:40.467544 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186bd0a-7cc6-4bcf-a599-5f26097f5ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:40.468699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186bd0a-7cc6-4bcf-a599-5f26097f5ceb admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:40.468996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186bd0a-7cc6-4bcf-a599-5f26097f5ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:40.469280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186bd0a-7cc6-4bcf-a599-5f26097f5ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:40.470197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186bd0a-7cc6-4bcf-a599-5f26097f5ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:40.470968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:40.472236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186bd0a-7cc6-4bcf-a599-5f26097f5ceb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:40.485582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1049/2136] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:02:40.489530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186bd0a-7cc6-4bcf-a599-5f26097f5ceb admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:40.505736 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186bd0a-7cc6-4bcf-a599-5f26097f5ceb admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:40.514122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aec457-6e7b-4706-bcc8-2b2d07d05f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:40.515169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aec457-6e7b-4706-bcc8-2b2d07d05f64 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:40.515693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aec457-6e7b-4706-bcc8-2b2d07d05f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:40.516258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aec457-6e7b-4706-bcc8-2b2d07d05f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:40.517639 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aec457-6e7b-4706-bcc8-2b2d07d05f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:40.519908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aec457-6e7b-4706-bcc8-2b2d07d05f64 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:40.539335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1050/2137] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:40.545029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aec457-6e7b-4706-bcc8-2b2d07d05f64 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:40.561943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aec457-6e7b-4706-bcc8-2b2d07d05f64 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:40.569127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c93d2-ae6c-4bc3-afc8-c3a8ff9572c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:40.570223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c93d2-ae6c-4bc3-afc8-c3a8ff9572c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:40.570717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c93d2-ae6c-4bc3-afc8-c3a8ff9572c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:40.571201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c93d2-ae6c-4bc3-afc8-c3a8ff9572c4 admin admin] PATH_INFO: `/v3/projects/2d91b39df422425d8e966ff1d23149a9/users/d81832021ecb4cf983fe91f979d8321a/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:40.599874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c93d2-ae6c-4bc3-afc8-c3a8ff9572c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d91b39df422425d8e966ff1d23149a9, user_id=d81832021ecb4cf983fe91f979d8321a, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:40.603076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c93d2-ae6c-4bc3-afc8-c3a8ff9572c4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:40.691839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1051/2138] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:40 2022] PUT /identity/v3/projects/2d91b39df422425d8e966ff1d23149a9/users/d81832021ecb4cf983fe91f979d8321a/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:40.697677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859089ce-1ad9-4684-8353-bdd6ca0565fd None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:40.698032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859089ce-1ad9-4684-8353-bdd6ca0565fd None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:40.698585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859089ce-1ad9-4684-8353-bdd6ca0565fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:40.704839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b723d57a7dc44b4d8f8af2a9f47c614f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:40.731754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b723d57a7dc44b4d8f8af2a9f47c614f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:40.759069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-859089ce-1ad9-4684-8353-bdd6ca0565fd None None] MFA Rules not processed for user `d81832021ecb4cf983fe91f979d8321a`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:40.762326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-859089ce-1ad9-4684-8353-bdd6ca0565fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:40.762773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b723d57a7dc44b4d8f8af2a9f47c614f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:40.763527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b723d57a7dc44b4d8f8af2a9f47c614f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:40.848130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6d4a6f16-6bca-4910-bc76-163a640244e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b723d57a7dc44b4d8f8af2a9f47c614f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:40.856351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1088/2139] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:40 2022] DELETE /identity/v3/users/b723d57a7dc44b4d8f8af2a9f47c614f => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:40.865442 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c645b5-03c4-48f9-9f69-1920ec196e1b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:40.865738 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c645b5-03c4-48f9-9f69-1920ec196e1b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:40.866023 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c645b5-03c4-48f9-9f69-1920ec196e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:40.873005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1052/2140] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:40.891092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-86c645b5-03c4-48f9-9f69-1920ec196e1b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:40.891326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859089ce-1ad9-4684-8353-bdd6ca0565fd None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:40.892610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-859089ce-1ad9-4684-8353-bdd6ca0565fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:40.896754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-86c645b5-03c4-48f9-9f69-1920ec196e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:40.977970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859089ce-1ad9-4684-8353-bdd6ca0565fd None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:41.009790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1089/2141] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:41.019090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:41.023023 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c645b5-03c4-48f9-9f69-1920ec196e1b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:41.036180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-86c645b5-03c4-48f9-9f69-1920ec196e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:41.064141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca33ef49ba44832a7fb45b2317d211c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:41.065260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:41.065705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:41.066104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:41.068139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:41.237532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:41.237532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:41.247893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1053/2142] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:41.261975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c645b5-03c4-48f9-9f69-1920ec196e1b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:41.273458 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d6c59-bd93-4c71-91f7-9cfe9ac5661b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:41.274339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d6c59-bd93-4c71-91f7-9cfe9ac5661b service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:41.286142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d6c59-bd93-4c71-91f7-9cfe9ac5661b service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:41.286142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d6c59-bd93-4c71-91f7-9cfe9ac5661b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:41.298657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d6c59-bd93-4c71-91f7-9cfe9ac5661b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:41.300147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d6c59-bd93-4c71-91f7-9cfe9ac5661b service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:41.311810 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1090/2143] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:41.877115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:41.878747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:41.938675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c1a33-e7ae-4716-99f1-2cc266d161ae service glance] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:41.956852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:41.958177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:41.959571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:41.960186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] PATH_INFO: `/v3/projects/ba0a6e1b9a2b43a097a0b25f913ad2d9` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:41.962374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0a6e1b9a2b43a097a0b25f913ad2d9)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:41.966106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:42.190246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0a6e1b9a2b43a097a0b25f913ad2d9', 'request_id': 'req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.200622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0a6e1b9a2b43a097a0b25f913ad2d9', 'request_id': 'req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.305568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0a6e1b9a2b43a097a0b25f913ad2d9', 'request_id': 'req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.321711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] The token cache is being invalidate because project ba0a6e1b9a2b43a097a0b25f913ad2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:42.322613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.325207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.331723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1054/2144] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:41 2022] DELETE /identity/v3/projects/ba0a6e1b9a2b43a097a0b25f913ad2d9 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:42.332985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d6c59-bd93-4c71-91f7-9cfe9ac5661b service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:42.385486 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d6c59-bd93-4c71-91f7-9cfe9ac5661b service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:42.527094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:42.528479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:42.529699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:42.530262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] PATH_INFO: `/v3/users/ef93fe8aebdf45e99b31ecd43c8c528f` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:42.533507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef93fe8aebdf45e99b31ecd43c8c528f)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:42.536088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:42.763411 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef93fe8aebdf45e99b31ecd43c8c528f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.793476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef93fe8aebdf45e99b31ecd43c8c528f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.815449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef93fe8aebdf45e99b31ecd43c8c528f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.815449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef93fe8aebdf45e99b31ecd43c8c528f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.897870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-726ab36a-0d5b-453b-ba1c-6d9735e05d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef93fe8aebdf45e99b31ecd43c8c528f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:42.918255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1091/2145] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:42 2022] DELETE /identity/v3/users/ef93fe8aebdf45e99b31ecd43c8c528f => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:43.122243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:43.185266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:44.333423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:44.335270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:44.380271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad5c58-9b1a-4a6b-a318-92e9a6cde8c1 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:44.522379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:44.526331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:44.526331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:44.526331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] PATH_INFO: `/v3/projects/c74d98d06f6c4b1b867d075217f0a708` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:44.526331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74d98d06f6c4b1b867d075217f0a708)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:44.529201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:44.665535 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74d98d06f6c4b1b867d075217f0a708', 'request_id': 'req-da5747e5-53c0-4e4b-8633-41477f35055b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:44.683320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74d98d06f6c4b1b867d075217f0a708', 'request_id': 'req-da5747e5-53c0-4e4b-8633-41477f35055b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:44.778855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74d98d06f6c4b1b867d075217f0a708', 'request_id': 'req-da5747e5-53c0-4e4b-8633-41477f35055b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:44.797936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] The token cache is being invalidate because project c74d98d06f6c4b1b867d075217f0a708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:44.798221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:44.800050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-da5747e5-53c0-4e4b-8633-41477f35055b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:44.802310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1055/2146] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:44 2022] DELETE /identity/v3/projects/c74d98d06f6c4b1b867d075217f0a708 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:44.807504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a0a9c-f307-417e-9df2-fa560ccd2119 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:44.808012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a0a9c-f307-417e-9df2-fa560ccd2119 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:44.808012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a0a9c-f307-417e-9df2-fa560ccd2119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:44.848617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d70a0a9c-f307-417e-9df2-fa560ccd2119 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:44.851080 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d70a0a9c-f307-417e-9df2-fa560ccd2119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:44.972151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1092/2147] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:44.985068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb739d6-f503-4d00-9b3e-629605d1441c None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:44.985608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb739d6-f503-4d00-9b3e-629605d1441c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:44.985895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb739d6-f503-4d00-9b3e-629605d1441c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:45.013962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-edb739d6-f503-4d00-9b3e-629605d1441c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:45.016942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-edb739d6-f503-4d00-9b3e-629605d1441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:45.034049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a0a9c-f307-417e-9df2-fa560ccd2119 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:45.045553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a0a9c-f307-417e-9df2-fa560ccd2119 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:45.049108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1056/2148] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:45.051619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb28ce0-0509-4840-a005-ab7e147da95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:45.054053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb28ce0-0509-4840-a005-ab7e147da95d admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:45.054568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb28ce0-0509-4840-a005-ab7e147da95d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:45.055018 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb28ce0-0509-4840-a005-ab7e147da95d admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:45.055992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb28ce0-0509-4840-a005-ab7e147da95d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:45.056468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb28ce0-0509-4840-a005-ab7e147da95d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:45.058664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb28ce0-0509-4840-a005-ab7e147da95d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:45.080287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb739d6-f503-4d00-9b3e-629605d1441c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:45.087498 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1093/2149] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:02:45.099130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb739d6-f503-4d00-9b3e-629605d1441c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:45.104857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a9f6d-7550-4c82-8e9a-ac0122e83448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:45.105805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a9f6d-7550-4c82-8e9a-ac0122e83448 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:45.106250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a9f6d-7550-4c82-8e9a-ac0122e83448 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:45.106730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a9f6d-7550-4c82-8e9a-ac0122e83448 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:45.107591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a9f6d-7550-4c82-8e9a-ac0122e83448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:45.109745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a9f6d-7550-4c82-8e9a-ac0122e83448 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:45.211820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1057/2150] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:45 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:45.216457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb28ce0-0509-4840-a005-ab7e147da95d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:45.227736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb28ce0-0509-4840-a005-ab7e147da95d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:45.382922 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447091c-495c-42b2-8530-8db197011de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:45.386776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447091c-495c-42b2-8530-8db197011de1 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:45.389046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447091c-495c-42b2-8530-8db197011de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:45.389046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447091c-495c-42b2-8530-8db197011de1 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:45.395909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447091c-495c-42b2-8530-8db197011de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:45.404473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447091c-495c-42b2-8530-8db197011de1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:45.544203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a9f6d-7550-4c82-8e9a-ac0122e83448 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:45.560749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1094/2151] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:45 2022] POST /identity/v3/users => generated 283 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:45.573273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a9f6d-7550-4c82-8e9a-ac0122e83448 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:45.579926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a42d74-f063-4646-b8d8-a16fb26e9fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:45.580869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a42d74-f063-4646-b8d8-a16fb26e9fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:45.581443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a42d74-f063-4646-b8d8-a16fb26e9fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:45.581897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a42d74-f063-4646-b8d8-a16fb26e9fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:45.584707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a42d74-f063-4646-b8d8-a16fb26e9fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:45.586633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a42d74-f063-4646-b8d8-a16fb26e9fa1 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:45.603824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447091c-495c-42b2-8530-8db197011de1 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:45.614663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1058/2152] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:45.620678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447091c-495c-42b2-8530-8db197011de1 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:45.627369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e5253-4839-49e7-adc6-5cab4a4cabd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:45.628181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e5253-4839-49e7-adc6-5cab4a4cabd8 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:45.628924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e5253-4839-49e7-adc6-5cab4a4cabd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:45.629787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e5253-4839-49e7-adc6-5cab4a4cabd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:45.630785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e5253-4839-49e7-adc6-5cab4a4cabd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:45.633197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e5253-4839-49e7-adc6-5cab4a4cabd8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:45.651230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1095/2153] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:45.652939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a42d74-f063-4646-b8d8-a16fb26e9fa1 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:45.669638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a42d74-f063-4646-b8d8-a16fb26e9fa1 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:45.675362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78ca73-78fa-47cb-81e3-977c7ddb58ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:45.676611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78ca73-78fa-47cb-81e3-977c7ddb58ff admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:45.677057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78ca73-78fa-47cb-81e3-977c7ddb58ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:45.677369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78ca73-78fa-47cb-81e3-977c7ddb58ff admin admin] PATH_INFO: `/v3/projects/8f5312241bd34492ba034eae13963f20/users/bea95157e08645ac9c38f7dd8700ed79/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:45.722125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78ca73-78fa-47cb-81e3-977c7ddb58ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5312241bd34492ba034eae13963f20, user_id=bea95157e08645ac9c38f7dd8700ed79, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:45.735625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78ca73-78fa-47cb-81e3-977c7ddb58ff admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:45.844967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:45.850484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1059/2154] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:45 2022] PUT /identity/v3/projects/8f5312241bd34492ba034eae13963f20/users/bea95157e08645ac9c38f7dd8700ed79/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:45.866602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074303-186c-4f54-a17e-e522fe3a4922 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:45.869244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074303-186c-4f54-a17e-e522fe3a4922 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:45.869891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074303-186c-4f54-a17e-e522fe3a4922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:45.917277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a8074303-186c-4f54-a17e-e522fe3a4922 None None] MFA Rules not processed for user `bea95157e08645ac9c38f7dd8700ed79`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:45.919230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a8074303-186c-4f54-a17e-e522fe3a4922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:45.970982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:46.038575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1096/2155] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:46.044860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78ca73-78fa-47cb-81e3-977c7ddb58ff admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:46.203814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78ca73-78fa-47cb-81e3-977c7ddb58ff admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:46.238242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015289e-587d-40ed-8dfc-5ee5fe572286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:46.241531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015289e-587d-40ed-8dfc-5ee5fe572286 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:46.242214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015289e-587d-40ed-8dfc-5ee5fe572286 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:46.242690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015289e-587d-40ed-8dfc-5ee5fe572286 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:46.244024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015289e-587d-40ed-8dfc-5ee5fe572286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:46.254372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015289e-587d-40ed-8dfc-5ee5fe572286 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:46.384694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1060/2156] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:46 2022] POST /identity/v3/projects => generated 365 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:46.388940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074303-186c-4f54-a17e-e522fe3a4922 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:46.413037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074303-186c-4f54-a17e-e522fe3a4922 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:46.571610 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e14b5-8a4a-4628-9c98-8a8fbcbf869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:46.574069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e14b5-8a4a-4628-9c98-8a8fbcbf869f admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:46.574365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e14b5-8a4a-4628-9c98-8a8fbcbf869f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:46.574820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e14b5-8a4a-4628-9c98-8a8fbcbf869f admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:46.582986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e14b5-8a4a-4628-9c98-8a8fbcbf869f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:46.589907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e14b5-8a4a-4628-9c98-8a8fbcbf869f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:46.641630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98e4f4-a3ee-4f4a-825c-3097983444be None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:46.642996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98e4f4-a3ee-4f4a-825c-3097983444be None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:46.643473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98e4f4-a3ee-4f4a-825c-3097983444be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:46.712454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-2d98e4f4-a3ee-4f4a-825c-3097983444be None None] MFA Rules not processed for user `822e56f086f0481ca4c6970adf85e396`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:46.714544 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2d98e4f4-a3ee-4f4a-825c-3097983444be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:46.731063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1097/2157] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:46 2022] POST /identity/v3/users => generated 289 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:46.734391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e14b5-8a4a-4628-9c98-8a8fbcbf869f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:46.794104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e14b5-8a4a-4628-9c98-8a8fbcbf869f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:46.810886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bba6e-e79c-4888-9206-e1e96c8a69f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:46.811660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bba6e-e79c-4888-9206-e1e96c8a69f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:46.814036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bba6e-e79c-4888-9206-e1e96c8a69f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:46.814036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bba6e-e79c-4888-9206-e1e96c8a69f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:46.818568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bba6e-e79c-4888-9206-e1e96c8a69f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:46.821497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bba6e-e79c-4888-9206-e1e96c8a69f6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:46.847753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1061/2158] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:46.860690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1098/2159] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:46.869615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98e4f4-a3ee-4f4a-825c-3097983444be None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:46.870021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bba6e-e79c-4888-9206-e1e96c8a69f6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:46.872681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c10bba6e-e79c-4888-9206-e1e96c8a69f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:46.886201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98e4f4-a3ee-4f4a-825c-3097983444be None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:46.905304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41aa48b-7c42-43a4-9f5c-e88798b69c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:46.906238 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41aa48b-7c42-43a4-9f5c-e88798b69c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:46.906523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41aa48b-7c42-43a4-9f5c-e88798b69c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:46.906937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41aa48b-7c42-43a4-9f5c-e88798b69c8d admin admin] PATH_INFO: `/v3/projects/6e3fd63409d94afd9bb94ea6a8c32690/users/33e19ba67f2a439395fdd7ea2bab4778/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:46.954617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41aa48b-7c42-43a4-9f5c-e88798b69c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3fd63409d94afd9bb94ea6a8c32690, user_id=33e19ba67f2a439395fdd7ea2bab4778, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:46.957669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41aa48b-7c42-43a4-9f5c-e88798b69c8d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:47.038918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bba6e-e79c-4888-9206-e1e96c8a69f6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:47.045764 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1062/2160] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:46 2022] PUT /identity/v3/projects/6e3fd63409d94afd9bb94ea6a8c32690/users/33e19ba67f2a439395fdd7ea2bab4778/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:47.046917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41aa48b-7c42-43a4-9f5c-e88798b69c8d admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:47.052037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f41aa48b-7c42-43a4-9f5c-e88798b69c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:47.167629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41aa48b-7c42-43a4-9f5c-e88798b69c8d admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:47.174418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b18197-3e79-4f04-9fe6-7791aad5d56e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:47.175479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b18197-3e79-4f04-9fe6-7791aad5d56e service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:47.175772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b18197-3e79-4f04-9fe6-7791aad5d56e service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:47.176296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b18197-3e79-4f04-9fe6-7791aad5d56e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:47.179672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:47.182580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:47.182869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:47.183147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] PATH_INFO: `/v3/users/51a44ebaf5a64f738cc6ad3d326b2ffb` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:47.187079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a44ebaf5a64f738cc6ad3d326b2ffb)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:47.191214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:47.310210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b18197-3e79-4f04-9fe6-7791aad5d56e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:47.312296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b18197-3e79-4f04-9fe6-7791aad5d56e service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:47.338025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1099/2161] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:47.341761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b18197-3e79-4f04-9fe6-7791aad5d56e service nova] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:47.351617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b18197-3e79-4f04-9fe6-7791aad5d56e service nova] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:47.358445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4058032-bb9a-4185-80ad-ba4ed1b07769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:47.360071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058032-bb9a-4185-80ad-ba4ed1b07769 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:47.360448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058032-bb9a-4185-80ad-ba4ed1b07769 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:47.360753 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058032-bb9a-4185-80ad-ba4ed1b07769 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:47.371008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4058032-bb9a-4185-80ad-ba4ed1b07769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:47.375303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4058032-bb9a-4185-80ad-ba4ed1b07769 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:47.388202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1100/2162] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:47.392304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4058032-bb9a-4185-80ad-ba4ed1b07769 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:47.418252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4058032-bb9a-4185-80ad-ba4ed1b07769 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:47.436188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375a1b8-158f-4236-ba4f-cc652eec6405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:47.437137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375a1b8-158f-4236-ba4f-cc652eec6405 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:47.437423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375a1b8-158f-4236-ba4f-cc652eec6405 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:47.437727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375a1b8-158f-4236-ba4f-cc652eec6405 admin admin] PATH_INFO: `/v3/projects/6e3fd63409d94afd9bb94ea6a8c32690/users/33e19ba67f2a439395fdd7ea2bab4778/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:47.448021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375a1b8-158f-4236-ba4f-cc652eec6405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3fd63409d94afd9bb94ea6a8c32690, user_id=33e19ba67f2a439395fdd7ea2bab4778, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:47.451229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375a1b8-158f-4236-ba4f-cc652eec6405 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:47.469828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1101/2163] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:47 2022] PUT /identity/v3/projects/6e3fd63409d94afd9bb94ea6a8c32690/users/33e19ba67f2a439395fdd7ea2bab4778/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:47.483115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bef3f8-2633-4af5-9c32-30337bdacec2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:47.483604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bef3f8-2633-4af5-9c32-30337bdacec2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:47.484088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bef3f8-2633-4af5-9c32-30337bdacec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:47.494999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a44ebaf5a64f738cc6ad3d326b2ffb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:47.506271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a44ebaf5a64f738cc6ad3d326b2ffb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:47.507417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a44ebaf5a64f738cc6ad3d326b2ffb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:47.533786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-47bef3f8-2633-4af5-9c32-30337bdacec2 None None] MFA Rules not processed for user `33e19ba67f2a439395fdd7ea2bab4778`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:47.536122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-47bef3f8-2633-4af5-9c32-30337bdacec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:47.625134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a44ebaf5a64f738cc6ad3d326b2ffb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:47.666983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4e82d5b3-ac36-4b84-9aa7-262317b09979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a44ebaf5a64f738cc6ad3d326b2ffb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:47.703827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1063/2164] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:47 2022] DELETE /identity/v3/users/51a44ebaf5a64f738cc6ad3d326b2ffb => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:47.706901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796216a8-4737-4df6-b2c7-22ad7079afe4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:47.711501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796216a8-4737-4df6-b2c7-22ad7079afe4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:47.711501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796216a8-4737-4df6-b2c7-22ad7079afe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:47.721268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1102/2165] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:47.766935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bef3f8-2633-4af5-9c32-30337bdacec2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:47.768323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-47bef3f8-2633-4af5-9c32-30337bdacec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:47.776919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-796216a8-4737-4df6-b2c7-22ad7079afe4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:47.780371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-796216a8-4737-4df6-b2c7-22ad7079afe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:47.874127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bef3f8-2633-4af5-9c32-30337bdacec2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:47.984993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1064/2166] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:48.006321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab734574-85b1-4f03-8866-ccacdafce131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:48.007496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab734574-85b1-4f03-8866-ccacdafce131 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:48.007934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab734574-85b1-4f03-8866-ccacdafce131 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:48.008399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab734574-85b1-4f03-8866-ccacdafce131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:48.009454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796216a8-4737-4df6-b2c7-22ad7079afe4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:48.015763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-796216a8-4737-4df6-b2c7-22ad7079afe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:48.153234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab734574-85b1-4f03-8866-ccacdafce131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:48.155514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab734574-85b1-4f03-8866-ccacdafce131 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:48.202771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796216a8-4737-4df6-b2c7-22ad7079afe4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:48.214775 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2bcd5-2d21-4c7b-8c26-11ab74371a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:48.215460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2bcd5-2d21-4c7b-8c26-11ab74371a28 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:48.215745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2bcd5-2d21-4c7b-8c26-11ab74371a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:48.216021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2bcd5-2d21-4c7b-8c26-11ab74371a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:48.221470 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1103/2167] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:48.242787 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2bcd5-2d21-4c7b-8c26-11ab74371a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:48.245008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2bcd5-2d21-4c7b-8c26-11ab74371a28 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:48.273573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1065/2168] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:48.417525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:48.860067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:49.601880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab734574-85b1-4f03-8866-ccacdafce131 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:49.604046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ab734574-85b1-4f03-8866-ccacdafce131 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:49.632361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab734574-85b1-4f03-8866-ccacdafce131 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:49.637313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:49.638404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:49.638842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:49.639406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] PATH_INFO: `/v3/users/bea95157e08645ac9c38f7dd8700ed79` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:49.641151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bea95157e08645ac9c38f7dd8700ed79)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:49.643545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:49.759302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2bcd5-2d21-4c7b-8c26-11ab74371a28 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:49.764678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ffb2bcd5-2d21-4c7b-8c26-11ab74371a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:49.791405 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2bcd5-2d21-4c7b-8c26-11ab74371a28 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:49.801147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:49.802309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:49.802779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:49.803201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] PATH_INFO: `/v3/projects/126709ddf9e6431cbb28317d0bd4de27` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:49.804724 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=126709ddf9e6431cbb28317d0bd4de27)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:49.808132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:49.906850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bea95157e08645ac9c38f7dd8700ed79'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:49.938107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bea95157e08645ac9c38f7dd8700ed79'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:49.952788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bea95157e08645ac9c38f7dd8700ed79'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:49.958246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bea95157e08645ac9c38f7dd8700ed79'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.022138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126709ddf9e6431cbb28317d0bd4de27', 'request_id': 'req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.030891 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126709ddf9e6431cbb28317d0bd4de27', 'request_id': 'req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.065981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-09120e8f-08e8-4172-9ca5-d4b785b5e2fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bea95157e08645ac9c38f7dd8700ed79'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.075343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1104/2169] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:49 2022] DELETE /identity/v3/users/bea95157e08645ac9c38f7dd8700ed79 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:50.081351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18633c79-fc04-4f90-bc9f-54b738aaf44b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:50.082011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18633c79-fc04-4f90-bc9f-54b738aaf44b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:50.082568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18633c79-fc04-4f90-bc9f-54b738aaf44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:50.110629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-18633c79-fc04-4f90-bc9f-54b738aaf44b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:50.112119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-18633c79-fc04-4f90-bc9f-54b738aaf44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:50.167350 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126709ddf9e6431cbb28317d0bd4de27', 'request_id': 'req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.226248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] The token cache is being invalidate because project 126709ddf9e6431cbb28317d0bd4de27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:50.226248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.226248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.243635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1066/2170] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:49 2022] DELETE /identity/v3/projects/126709ddf9e6431cbb28317d0bd4de27 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:50.245711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:50.247728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1105/2171] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:50.254553 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:50.276830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18633c79-fc04-4f90-bc9f-54b738aaf44b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:50.279030 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-18633c79-fc04-4f90-bc9f-54b738aaf44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:50.419263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:50.458013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18633c79-fc04-4f90-bc9f-54b738aaf44b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:50.474186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f47f9-df33-4190-8c07-f9db1049b06f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:50.475276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f47f9-df33-4190-8c07-f9db1049b06f service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:50.475570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f47f9-df33-4190-8c07-f9db1049b06f service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:50.475898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f47f9-df33-4190-8c07-f9db1049b06f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:50.506672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f47f9-df33-4190-8c07-f9db1049b06f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:50.512080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f991c-ea3e-4b5d-a2b0-180aa28912f4 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:50.526489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:50.527563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f47f9-df33-4190-8c07-f9db1049b06f service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:50.532408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:50.533215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:50.533933 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] PATH_INFO: `/v3/users/265f2c7eaf5a408f9ed96db672d3d2e8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:50.536581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=265f2c7eaf5a408f9ed96db672d3d2e8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:50.544307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:50.615204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1106/2172] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:02:50.796952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:50.872867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '265f2c7eaf5a408f9ed96db672d3d2e8'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.901327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '265f2c7eaf5a408f9ed96db672d3d2e8'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.907191 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '265f2c7eaf5a408f9ed96db672d3d2e8'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:50.928755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1526da-e46c-4605-8820-143650cac8b8 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:50.929106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1526da-e46c-4605-8820-143650cac8b8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:50.929318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1526da-e46c-4605-8820-143650cac8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:50.964602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2a1526da-e46c-4605-8820-143650cac8b8 None None] MFA Rules not processed for user `3f1b6145d67d4cbf9dcd398736e6b0da`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:50.965949 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2a1526da-e46c-4605-8820-143650cac8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:50.996398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '265f2c7eaf5a408f9ed96db672d3d2e8'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:51.043779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '265f2c7eaf5a408f9ed96db672d3d2e8'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:51.062551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1067/2173] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:50 2022] DELETE /identity/v3/users/265f2c7eaf5a408f9ed96db672d3d2e8 => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:51.138880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1107/2174] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:51.143505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:51.145681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:51.288389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920df01-db98-472b-8030-084c2e8986f3 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:51.302056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82eb66a-1c40-488f-95b2-ae5852931c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:51.302762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82eb66a-1c40-488f-95b2-ae5852931c4c admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:51.303051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82eb66a-1c40-488f-95b2-ae5852931c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:51.303332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82eb66a-1c40-488f-95b2-ae5852931c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:51.304251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82eb66a-1c40-488f-95b2-ae5852931c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:51.307108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82eb66a-1c40-488f-95b2-ae5852931c4c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:51.436689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1068/2175] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:51 2022] POST /identity/v3/projects => generated 383 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:51.441568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1526da-e46c-4605-8820-143650cac8b8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:51.459337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1526da-e46c-4605-8820-143650cac8b8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:51.612619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d02f53-3fc5-44ac-a136-46c669c8e476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:51.616514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d02f53-3fc5-44ac-a136-46c669c8e476 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:51.617154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d02f53-3fc5-44ac-a136-46c669c8e476 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:51.617264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d02f53-3fc5-44ac-a136-46c669c8e476 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:51.619099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d02f53-3fc5-44ac-a136-46c669c8e476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:51.621482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d02f53-3fc5-44ac-a136-46c669c8e476 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:51.636140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9d18d-2bb7-407b-8a29-ba1b4c872ac1 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:51.636140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9d18d-2bb7-407b-8a29-ba1b4c872ac1 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:51.636446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9d18d-2bb7-407b-8a29-ba1b4c872ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:51.676307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-03d9d18d-2bb7-407b-8a29-ba1b4c872ac1 None None] MFA Rules not processed for user `822e56f086f0481ca4c6970adf85e396`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:51.677592 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-03d9d18d-2bb7-407b-8a29-ba1b4c872ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:51.803217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1069/2176] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:51.804409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9d18d-2bb7-407b-8a29-ba1b4c872ac1 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:51.809856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-03d9d18d-2bb7-407b-8a29-ba1b4c872ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:51.825530 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1108/2177] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:51 2022] POST /identity/v3/users => generated 298 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:51.828090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d02f53-3fc5-44ac-a136-46c669c8e476 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:51.850792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d02f53-3fc5-44ac-a136-46c669c8e476 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:51.858805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9d18d-2bb7-407b-8a29-ba1b4c872ac1 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:51.867575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49b4d3-ff8e-4e1a-b73e-f469ed77b5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:51.870370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49b4d3-ff8e-4e1a-b73e-f469ed77b5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:51.873213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49b4d3-ff8e-4e1a-b73e-f469ed77b5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:51.874558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49b4d3-ff8e-4e1a-b73e-f469ed77b5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:51.876593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49b4d3-ff8e-4e1a-b73e-f469ed77b5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:51.877218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:51.879784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:51.880195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:51.880674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49b4d3-ff8e-4e1a-b73e-f469ed77b5d1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:51.881060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] PATH_INFO: `/v3/projects/8f5312241bd34492ba034eae13963f20` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:51.884188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5312241bd34492ba034eae13963f20)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:51.890386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:51.927930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1109/2178] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:51.928905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49b4d3-ff8e-4e1a-b73e-f469ed77b5d1 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:51.934698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8b49b4d3-ff8e-4e1a-b73e-f469ed77b5d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:51.971028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49b4d3-ff8e-4e1a-b73e-f469ed77b5d1 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:51.980215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:51.981133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:51.981425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:51.981708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] PATH_INFO: `/v3/projects/e4a804a7fd4645c8ae05d67a75cbc8e3` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:51.983338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a804a7fd4645c8ae05d67a75cbc8e3)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:51.985016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:52.013684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:52.121205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5312241bd34492ba034eae13963f20', 'request_id': 'req-977fa86b-8a75-4ad3-a30a-1543a57790ce'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.142495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a804a7fd4645c8ae05d67a75cbc8e3', 'request_id': 'req-0d901aa6-bba9-4481-93aa-05be0486384f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.162451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5312241bd34492ba034eae13963f20', 'request_id': 'req-977fa86b-8a75-4ad3-a30a-1543a57790ce'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.183078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a804a7fd4645c8ae05d67a75cbc8e3', 'request_id': 'req-0d901aa6-bba9-4481-93aa-05be0486384f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.196861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a804a7fd4645c8ae05d67a75cbc8e3', 'request_id': 'req-0d901aa6-bba9-4481-93aa-05be0486384f'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.293389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5312241bd34492ba034eae13963f20', 'request_id': 'req-977fa86b-8a75-4ad3-a30a-1543a57790ce'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.319427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] The token cache is being invalidate because project 8f5312241bd34492ba034eae13963f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:52.319747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.323209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.337686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1070/2179] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:51 2022] DELETE /identity/v3/projects/8f5312241bd34492ba034eae13963f20 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:52.343884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:52.351373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:52.444919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] The token cache is being invalidate because project e4a804a7fd4645c8ae05d67a75cbc8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:52.445224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.447315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.449656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1110/2180] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:51 2022] DELETE /identity/v3/projects/e4a804a7fd4645c8ae05d67a75cbc8e3 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:52.450577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:52.452271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:52.536722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fa86b-8a75-4ad3-a30a-1543a57790ce admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:52.616036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d901aa6-bba9-4481-93aa-05be0486384f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:52.624149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:52.629455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:52.629455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:52.629455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] PATH_INFO: `/v3/users/33e19ba67f2a439395fdd7ea2bab4778` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:52.629455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e19ba67f2a439395fdd7ea2bab4778)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:52.632051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:52.663287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdaed2-aa4f-40e8-8805-c7750cb4a1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:52.664103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdaed2-aa4f-40e8-8805-c7750cb4a1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:52.664402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdaed2-aa4f-40e8-8805-c7750cb4a1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:52.664787 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdaed2-aa4f-40e8-8805-c7750cb4a1dd admin admin] PATH_INFO: `/v3/projects/63a05ee2b3fe48ddbac52390d653027a/users/019c27e5d7564e82a2c666cbc87c378e/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:52.703546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdaed2-aa4f-40e8-8805-c7750cb4a1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a05ee2b3fe48ddbac52390d653027a, user_id=019c27e5d7564e82a2c666cbc87c378e, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:52.706583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdaed2-aa4f-40e8-8805-c7750cb4a1dd admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:52.798171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1071/2181] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:52 2022] PUT /identity/v3/projects/63a05ee2b3fe48ddbac52390d653027a/users/019c27e5d7564e82a2c666cbc87c378e/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:52.799515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3740285-eb5f-46c8-9363-17249ea3fc77 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:52.799813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3740285-eb5f-46c8-9363-17249ea3fc77 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:52.800096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3740285-eb5f-46c8-9363-17249ea3fc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:52.834020 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d3740285-eb5f-46c8-9363-17249ea3fc77 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:52.835624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d3740285-eb5f-46c8-9363-17249ea3fc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:52.917161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e19ba67f2a439395fdd7ea2bab4778'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.953627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e19ba67f2a439395fdd7ea2bab4778'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.972033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e19ba67f2a439395fdd7ea2bab4778'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:52.973134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e19ba67f2a439395fdd7ea2bab4778'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:53.027555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1072/2182] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:53.029808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3740285-eb5f-46c8-9363-17249ea3fc77 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:53.039555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d3740285-eb5f-46c8-9363-17249ea3fc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:53.069394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3740285-eb5f-46c8-9363-17249ea3fc77 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:53.076616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-bca21735-7043-43c4-b772-71357dab58bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e19ba67f2a439395fdd7ea2bab4778'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:53.081786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:53.083988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b3748-1d47-412e-a7bc-dd7fc2f03bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:53.086179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b3748-1d47-412e-a7bc-dd7fc2f03bbe admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:53.086458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b3748-1d47-412e-a7bc-dd7fc2f03bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:53.089275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b3748-1d47-412e-a7bc-dd7fc2f03bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:53.090101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1111/2183] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:52 2022] DELETE /identity/v3/users/33e19ba67f2a439395fdd7ea2bab4778 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:53.092365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b3748-1d47-412e-a7bc-dd7fc2f03bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:53.092518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b4087-b500-4e7f-962e-e1d3ce2122da None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:53.092518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b4087-b500-4e7f-962e-e1d3ce2122da None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:53.092518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b4087-b500-4e7f-962e-e1d3ce2122da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:53.107820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b3748-1d47-412e-a7bc-dd7fc2f03bbe admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:53.122643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-916b4087-b500-4e7f-962e-e1d3ce2122da None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:53.123947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-916b4087-b500-4e7f-962e-e1d3ce2122da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:53.145134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1073/2184] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:53.151129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b3748-1d47-412e-a7bc-dd7fc2f03bbe admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:53.160313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-295b3748-1d47-412e-a7bc-dd7fc2f03bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:53.185046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b3748-1d47-412e-a7bc-dd7fc2f03bbe admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:53.333419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1112/2185] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:53.337375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b4087-b500-4e7f-962e-e1d3ce2122da None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:53.349479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b4087-b500-4e7f-962e-e1d3ce2122da None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:53.356921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d1dac-1b6d-4ace-89be-85e6d349567f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:53.358954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d1dac-1b6d-4ace-89be-85e6d349567f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:53.363256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d1dac-1b6d-4ace-89be-85e6d349567f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:53.363256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d1dac-1b6d-4ace-89be-85e6d349567f admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:53.363256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d1dac-1b6d-4ace-89be-85e6d349567f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:53.363256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d1dac-1b6d-4ace-89be-85e6d349567f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:53.363864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d1dac-1b6d-4ace-89be-85e6d349567f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:53.380275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428ea3b-a85a-4a6d-adac-e63c8aba0cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:53.381225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428ea3b-a85a-4a6d-adac-e63c8aba0cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:53.381565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428ea3b-a85a-4a6d-adac-e63c8aba0cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:53.386293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428ea3b-a85a-4a6d-adac-e63c8aba0cbe admin admin] PATH_INFO: `/v3/projects/63a05ee2b3fe48ddbac52390d653027a/users/019c27e5d7564e82a2c666cbc87c378e/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:53.386595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1113/2186] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:02:53.388004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428ea3b-a85a-4a6d-adac-e63c8aba0cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a05ee2b3fe48ddbac52390d653027a, user_id=019c27e5d7564e82a2c666cbc87c378e, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:53.388522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d1dac-1b6d-4ace-89be-85e6d349567f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:53.391212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428ea3b-a85a-4a6d-adac-e63c8aba0cbe admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:53.403528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d1dac-1b6d-4ace-89be-85e6d349567f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:53.415163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1074/2187] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:53 2022] PUT /identity/v3/projects/63a05ee2b3fe48ddbac52390d653027a/users/019c27e5d7564e82a2c666cbc87c378e/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:53.493363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31126e-5618-4ebb-9ec9-00df93cefc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:53.494284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31126e-5618-4ebb-9ec9-00df93cefc0e admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:53.494739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31126e-5618-4ebb-9ec9-00df93cefc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:53.495223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31126e-5618-4ebb-9ec9-00df93cefc0e admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:53.496018 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31126e-5618-4ebb-9ec9-00df93cefc0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:53.498819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31126e-5618-4ebb-9ec9-00df93cefc0e admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:53.657834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1114/2188] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:53 2022] POST /identity/v3/projects => generated 385 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:53.661529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428ea3b-a85a-4a6d-adac-e63c8aba0cbe admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:53.675840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428ea3b-a85a-4a6d-adac-e63c8aba0cbe admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:53.762905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730d61a-db14-4aa3-9dfd-b6e69d75a883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:53.763954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730d61a-db14-4aa3-9dfd-b6e69d75a883 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:53.764317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730d61a-db14-4aa3-9dfd-b6e69d75a883 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:53.764741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730d61a-db14-4aa3-9dfd-b6e69d75a883 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:53.772169 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730d61a-db14-4aa3-9dfd-b6e69d75a883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:53.774587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730d61a-db14-4aa3-9dfd-b6e69d75a883 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:53.940490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1075/2189] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:53 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:53.944872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31126e-5618-4ebb-9ec9-00df93cefc0e admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:53.962412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31126e-5618-4ebb-9ec9-00df93cefc0e admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:53.983504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc01d-1d24-4b2f-8591-b0f3b9099b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:53.994281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc01d-1d24-4b2f-8591-b0f3b9099b93 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:53.994281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc01d-1d24-4b2f-8591-b0f3b9099b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:53.994281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc01d-1d24-4b2f-8591-b0f3b9099b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:53.994281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc01d-1d24-4b2f-8591-b0f3b9099b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:53.994281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc01d-1d24-4b2f-8591-b0f3b9099b93 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:54.012247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1115/2190] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:54.015450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730d61a-db14-4aa3-9dfd-b6e69d75a883 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:54.037106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730d61a-db14-4aa3-9dfd-b6e69d75a883 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:54.058514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14a8ee-d5de-46b3-abde-a1c28cb93c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:54.060047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14a8ee-d5de-46b3-abde-a1c28cb93c86 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:54.060047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14a8ee-d5de-46b3-abde-a1c28cb93c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:54.060047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14a8ee-d5de-46b3-abde-a1c28cb93c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:54.060989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14a8ee-d5de-46b3-abde-a1c28cb93c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:54.063667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14a8ee-d5de-46b3-abde-a1c28cb93c86 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:54.094640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1076/2191] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:54.096069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc01d-1d24-4b2f-8591-b0f3b9099b93 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:54.116243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc01d-1d24-4b2f-8591-b0f3b9099b93 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:54.122485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a456d4-511a-4680-b2fc-48936251d734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:54.123540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a456d4-511a-4680-b2fc-48936251d734 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:54.123983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a456d4-511a-4680-b2fc-48936251d734 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:54.124363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a456d4-511a-4680-b2fc-48936251d734 admin admin] PATH_INFO: `/v3/projects/2bb362ee74e1448b91a735db39b4ac18/users/3174a496203248f38d55e640bcc2cf5b/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:54.156851 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a456d4-511a-4680-b2fc-48936251d734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb362ee74e1448b91a735db39b4ac18, user_id=3174a496203248f38d55e640bcc2cf5b, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:54.162116 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a456d4-511a-4680-b2fc-48936251d734 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:54.268243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1116/2192] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:54 2022] PUT /identity/v3/projects/2bb362ee74e1448b91a735db39b4ac18/users/3174a496203248f38d55e640bcc2cf5b/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:54.288242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a8091-46c3-4414-9057-bf558c15ecdb None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:54.289238 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a8091-46c3-4414-9057-bf558c15ecdb None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:54.290321 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a8091-46c3-4414-9057-bf558c15ecdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:54.363382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-359a8091-46c3-4414-9057-bf558c15ecdb None None] MFA Rules not processed for user `3174a496203248f38d55e640bcc2cf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:54.377624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:54.378853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-359a8091-46c3-4414-9057-bf558c15ecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:54.647691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1077/2193] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:54.648144 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a456d4-511a-4680-b2fc-48936251d734 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:54.648412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a8091-46c3-4414-9057-bf558c15ecdb None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:54.648508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-11a456d4-511a-4680-b2fc-48936251d734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:54.852336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a8091-46c3-4414-9057-bf558c15ecdb None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:54.875064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdecbd-2117-4400-a830-c4e786a4e75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:54.885702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdecbd-2117-4400-a830-c4e786a4e75a admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:54.885702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdecbd-2117-4400-a830-c4e786a4e75a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:54.885702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdecbd-2117-4400-a830-c4e786a4e75a admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:54.885702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdecbd-2117-4400-a830-c4e786a4e75a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:54.885702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdecbd-2117-4400-a830-c4e786a4e75a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:54.890012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:54.956316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a456d4-511a-4680-b2fc-48936251d734 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:54.962723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:54.963941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:54.964431 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:54.964968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] PATH_INFO: `/v3/projects/6e3fd63409d94afd9bb94ea6a8c32690` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:54.969114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3fd63409d94afd9bb94ea6a8c32690)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:54.975042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:55.082489 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1078/2194] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:54 2022] POST /identity/v3/projects => generated 385 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:55.087418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdecbd-2117-4400-a830-c4e786a4e75a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:55.215131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3fd63409d94afd9bb94ea6a8c32690', 'request_id': 'req-1eef4c79-8abd-4518-b93e-8283c0e264c9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:55.233341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdecbd-2117-4400-a830-c4e786a4e75a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:55.245298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3fd63409d94afd9bb94ea6a8c32690', 'request_id': 'req-1eef4c79-8abd-4518-b93e-8283c0e264c9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:55.263292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3fd63409d94afd9bb94ea6a8c32690', 'request_id': 'req-1eef4c79-8abd-4518-b93e-8283c0e264c9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:55.337741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c4b8c-840f-44d4-8f16-8056f255e734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:55.338514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c4b8c-840f-44d4-8f16-8056f255e734 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:55.338865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c4b8c-840f-44d4-8f16-8056f255e734 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:55.339800 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c4b8c-840f-44d4-8f16-8056f255e734 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:55.340609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c4b8c-840f-44d4-8f16-8056f255e734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:55.342877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c4b8c-840f-44d4-8f16-8056f255e734 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:55.375542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] The token cache is being invalidate because project 6e3fd63409d94afd9bb94ea6a8c32690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:02:55.386680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:55.389260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1eef4c79-8abd-4518-b93e-8283c0e264c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:55.400456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1117/2195] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:02:54 2022] DELETE /identity/v3/projects/6e3fd63409d94afd9bb94ea6a8c32690 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:55.455168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1079/2196] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:55 2022] POST /identity/v3/users => generated 293 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:55.462951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d02b4-15ce-45c8-8d99-bd1c7fb42368 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:55.466888 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c4b8c-840f-44d4-8f16-8056f255e734 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:55.467327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d02b4-15ce-45c8-8d99-bd1c7fb42368 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:55.470536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d02b4-15ce-45c8-8d99-bd1c7fb42368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:55.494752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7c8c4b8c-840f-44d4-8f16-8056f255e734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:55.499396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d22d02b4-15ce-45c8-8d99-bd1c7fb42368 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:55.501118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d22d02b4-15ce-45c8-8d99-bd1c7fb42368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:55.529966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1118/2197] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:55.533204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92946e-3ab9-4d1f-bcad-6a62dfec12dc None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:55.533711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92946e-3ab9-4d1f-bcad-6a62dfec12dc None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:55.534243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92946e-3ab9-4d1f-bcad-6a62dfec12dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:55.558313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ae92946e-3ab9-4d1f-bcad-6a62dfec12dc None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:55.561017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ae92946e-3ab9-4d1f-bcad-6a62dfec12dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:55.567208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c4b8c-840f-44d4-8f16-8056f255e734 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:55.581771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1119/2198] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:55.582799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92946e-3ab9-4d1f-bcad-6a62dfec12dc None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:55.586988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4df761-f8ba-4bb3-851a-2cbc6fb2140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:55.593080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4df761-f8ba-4bb3-851a-2cbc6fb2140a admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:55.593080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4df761-f8ba-4bb3-851a-2cbc6fb2140a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:55.593080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4df761-f8ba-4bb3-851a-2cbc6fb2140a admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:55.593080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4df761-f8ba-4bb3-851a-2cbc6fb2140a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:55.595890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92946e-3ab9-4d1f-bcad-6a62dfec12dc None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:55.606317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4df761-f8ba-4bb3-851a-2cbc6fb2140a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:55.609665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e77222-2904-41f4-a40d-b8e68c03923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:55.610715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e77222-2904-41f4-a40d-b8e68c03923f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:55.611263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e77222-2904-41f4-a40d-b8e68c03923f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:55.611946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e77222-2904-41f4-a40d-b8e68c03923f admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:55.613430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e77222-2904-41f4-a40d-b8e68c03923f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:02:55.614096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e77222-2904-41f4-a40d-b8e68c03923f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:55.616384 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e77222-2904-41f4-a40d-b8e68c03923f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:55.632156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1080/2199] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:55.633388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4df761-f8ba-4bb3-851a-2cbc6fb2140a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:55.641428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1120/2200] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:02:55.646288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e77222-2904-41f4-a40d-b8e68c03923f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:55.653261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4df761-f8ba-4bb3-851a-2cbc6fb2140a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:55.662518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e77222-2904-41f4-a40d-b8e68c03923f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:55.667051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946327c-6f9a-4e4b-9e74-d9c34e14ba8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:55.668037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946327c-6f9a-4e4b-9e74-d9c34e14ba8c admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:55.668570 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946327c-6f9a-4e4b-9e74-d9c34e14ba8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:55.669087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946327c-6f9a-4e4b-9e74-d9c34e14ba8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:55.670132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946327c-6f9a-4e4b-9e74-d9c34e14ba8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:55.672551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946327c-6f9a-4e4b-9e74-d9c34e14ba8c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:55.673741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdbb67-4790-4be5-b69e-56c71cf9fccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:55.674583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdbb67-4790-4be5-b69e-56c71cf9fccd admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:55.674864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdbb67-4790-4be5-b69e-56c71cf9fccd admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:55.675390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdbb67-4790-4be5-b69e-56c71cf9fccd admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:55.676673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdbb67-4790-4be5-b69e-56c71cf9fccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:55.687791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdbb67-4790-4be5-b69e-56c71cf9fccd admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:55.693486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1081/2201] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:55.694718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946327c-6f9a-4e4b-9e74-d9c34e14ba8c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:55.707503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946327c-6f9a-4e4b-9e74-d9c34e14ba8c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:55.713961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236ee42-af75-4632-8f7c-036ea0d077aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:55.716725 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236ee42-af75-4632-8f7c-036ea0d077aa admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:55.722166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236ee42-af75-4632-8f7c-036ea0d077aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:55.722166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236ee42-af75-4632-8f7c-036ea0d077aa admin admin] PATH_INFO: `/v3/projects/ef61d7d231934295b3584ccdddf50b26/users/cfc4e417c4c146adacb1927dc0e875e6/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:55.741369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236ee42-af75-4632-8f7c-036ea0d077aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef61d7d231934295b3584ccdddf50b26, user_id=cfc4e417c4c146adacb1927dc0e875e6, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:55.744319 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236ee42-af75-4632-8f7c-036ea0d077aa admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:55.862680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1082/2202] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:55 2022] PUT /identity/v3/projects/ef61d7d231934295b3584ccdddf50b26/users/cfc4e417c4c146adacb1927dc0e875e6/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:02:55.866278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1121/2203] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:55 2022] POST /identity/v3/projects => generated 381 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:55.867317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236ee42-af75-4632-8f7c-036ea0d077aa admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:55.959596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a78f7-bec4-4d21-886d-3819b3507a0b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:55.960154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a78f7-bec4-4d21-886d-3819b3507a0b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:55.960670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a78f7-bec4-4d21-886d-3819b3507a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.018082 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-153a78f7-bec4-4d21-886d-3819b3507a0b None None] MFA Rules not processed for user `cfc4e417c4c146adacb1927dc0e875e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:56.019338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-153a78f7-bec4-4d21-886d-3819b3507a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:56.027049 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236ee42-af75-4632-8f7c-036ea0d077aa admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:56.170866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1122/2204] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:02:56.175946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a78f7-bec4-4d21-886d-3819b3507a0b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:56.184492 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46767dc9-d727-4e25-a87e-61c814115fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:56.184492 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46767dc9-d727-4e25-a87e-61c814115fdb admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.184492 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46767dc9-d727-4e25-a87e-61c814115fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.184492 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46767dc9-d727-4e25-a87e-61c814115fdb admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.188344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46767dc9-d727-4e25-a87e-61c814115fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:56.193631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46767dc9-d727-4e25-a87e-61c814115fdb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:56.198672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a78f7-bec4-4d21-886d-3819b3507a0b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:56.209664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af0e53-f6d7-41b1-8945-ef4a71bee87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:56.210641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af0e53-f6d7-41b1-8945-ef4a71bee87c admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.211114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af0e53-f6d7-41b1-8945-ef4a71bee87c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.213128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af0e53-f6d7-41b1-8945-ef4a71bee87c admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.215742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af0e53-f6d7-41b1-8945-ef4a71bee87c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:56.216866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af0e53-f6d7-41b1-8945-ef4a71bee87c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:56.330122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1083/2205] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:55 2022] POST /identity/v3/users => generated 291 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:56.335526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46767dc9-d727-4e25-a87e-61c814115fdb admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:56.351090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1123/2206] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:56 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:56.352343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af0e53-f6d7-41b1-8945-ef4a71bee87c admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:56.353962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46767dc9-d727-4e25-a87e-61c814115fdb admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:56.454592 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851ea4d-42be-49e1-97ad-bc5a5a846a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:56.455204 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851ea4d-42be-49e1-97ad-bc5a5a846a33 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.455492 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851ea4d-42be-49e1-97ad-bc5a5a846a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.455773 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851ea4d-42be-49e1-97ad-bc5a5a846a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.456521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851ea4d-42be-49e1-97ad-bc5a5a846a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:56.458229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851ea4d-42be-49e1-97ad-bc5a5a846a33 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:56.474306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1084/2207] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:02:56.478379 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851ea4d-42be-49e1-97ad-bc5a5a846a33 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:56.481766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af0e53-f6d7-41b1-8945-ef4a71bee87c admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:56.487181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a79bc-a246-421a-919b-31a0159f6626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:56.487892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a79bc-a246-421a-919b-31a0159f6626 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.488181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a79bc-a246-421a-919b-31a0159f6626 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.488568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a79bc-a246-421a-919b-31a0159f6626 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.489296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a79bc-a246-421a-919b-31a0159f6626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:56.492166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a79bc-a246-421a-919b-31a0159f6626 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:56.498153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851ea4d-42be-49e1-97ad-bc5a5a846a33 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:56.505872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71560a21-9d2c-48d1-96e5-03fbb377bec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:56.506567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71560a21-9d2c-48d1-96e5-03fbb377bec4 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.506936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71560a21-9d2c-48d1-96e5-03fbb377bec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.507224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71560a21-9d2c-48d1-96e5-03fbb377bec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.507998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71560a21-9d2c-48d1-96e5-03fbb377bec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:56.510316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71560a21-9d2c-48d1-96e5-03fbb377bec4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:56.521962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1085/2208] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:56.530629 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71560a21-9d2c-48d1-96e5-03fbb377bec4 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:56.541309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71560a21-9d2c-48d1-96e5-03fbb377bec4 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:56.546868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a14a8b-363d-4868-a6a8-78828b89139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:56.548069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14a8b-363d-4868-a6a8-78828b89139e admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.548517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14a8b-363d-4868-a6a8-78828b89139e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.548988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14a8b-363d-4868-a6a8-78828b89139e admin admin] PATH_INFO: `/v3/projects/01fbbc85cfbe461ca8ce7685281154b6/users/b56a2b4385b7457a88d6c0cb907080df/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.620261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a14a8b-363d-4868-a6a8-78828b89139e admin admin] RBAC: Authorizing `identity:create_grant(project_id=01fbbc85cfbe461ca8ce7685281154b6, user_id=b56a2b4385b7457a88d6c0cb907080df, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:56.626378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1124/2209] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:56 2022] POST /identity/v3/users => generated 299 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:56.627607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a79bc-a246-421a-919b-31a0159f6626 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:56.635390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a14a8b-363d-4868-a6a8-78828b89139e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:56.645682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a79bc-a246-421a-919b-31a0159f6626 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:56.650969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c075a17-0a85-486e-ad5e-7009f73ccf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:56.651995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c075a17-0a85-486e-ad5e-7009f73ccf3f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.652361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c075a17-0a85-486e-ad5e-7009f73ccf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.652744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c075a17-0a85-486e-ad5e-7009f73ccf3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.653574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c075a17-0a85-486e-ad5e-7009f73ccf3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:56.655884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c075a17-0a85-486e-ad5e-7009f73ccf3f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:56.700141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1125/2210] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:02:56.704084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c075a17-0a85-486e-ad5e-7009f73ccf3f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:56.729910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c075a17-0a85-486e-ad5e-7009f73ccf3f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:56.746450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62d1b3-9441-44b6-9290-518619605df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:56.747715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62d1b3-9441-44b6-9290-518619605df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.748177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62d1b3-9441-44b6-9290-518619605df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.749015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62d1b3-9441-44b6-9290-518619605df9 admin admin] PATH_INFO: `/v3/projects/60dd86f5ec50424c9426cbb039ffc1e5/users/083f1b8c7d5247ac8655bcf8b5564bf3/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.772713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62d1b3-9441-44b6-9290-518619605df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60dd86f5ec50424c9426cbb039ffc1e5, user_id=083f1b8c7d5247ac8655bcf8b5564bf3, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:56.784428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62d1b3-9441-44b6-9290-518619605df9 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:56.833059 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1086/2211] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:56 2022] PUT /identity/v3/projects/01fbbc85cfbe461ca8ce7685281154b6/users/b56a2b4385b7457a88d6c0cb907080df/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:56.844742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632e203-b101-44e8-8ebc-4a25018ffa5d None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.845591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632e203-b101-44e8-8ebc-4a25018ffa5d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.846829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632e203-b101-44e8-8ebc-4a25018ffa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.864289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1126/2212] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:56 2022] PUT /identity/v3/projects/60dd86f5ec50424c9426cbb039ffc1e5/users/083f1b8c7d5247ac8655bcf8b5564bf3/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:56.873211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62d1b3-9441-44b6-9290-518619605df9 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:56.895513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3632e203-b101-44e8-8ebc-4a25018ffa5d None None] MFA Rules not processed for user `b56a2b4385b7457a88d6c0cb907080df`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:56.898092 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3632e203-b101-44e8-8ebc-4a25018ffa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:56.978165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62d1b3-9441-44b6-9290-518619605df9 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:56.983210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5965c8-abc9-4b2a-aa4f-a04a173bad05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:56.984089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5965c8-abc9-4b2a-aa4f-a04a173bad05 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:56.984424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5965c8-abc9-4b2a-aa4f-a04a173bad05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:56.984842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5965c8-abc9-4b2a-aa4f-a04a173bad05 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:56.985730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5965c8-abc9-4b2a-aa4f-a04a173bad05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:56.987453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5965c8-abc9-4b2a-aa4f-a04a173bad05 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:57.003173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1127/2213] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:57.005931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5965c8-abc9-4b2a-aa4f-a04a173bad05 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:57.020300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5965c8-abc9-4b2a-aa4f-a04a173bad05 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:57.029293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156c6f6-25dd-470d-a91a-708bca334cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:57.030830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c6f6-25dd-470d-a91a-708bca334cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:57.031285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c6f6-25dd-470d-a91a-708bca334cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:57.031719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c6f6-25dd-470d-a91a-708bca334cdc admin admin] PATH_INFO: `/v3/projects/60dd86f5ec50424c9426cbb039ffc1e5/users/083f1b8c7d5247ac8655bcf8b5564bf3/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:57.035235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156c6f6-25dd-470d-a91a-708bca334cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=60dd86f5ec50424c9426cbb039ffc1e5, user_id=083f1b8c7d5247ac8655bcf8b5564bf3, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:57.038349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156c6f6-25dd-470d-a91a-708bca334cdc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:57.061159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1087/2214] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:57.065106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632e203-b101-44e8-8ebc-4a25018ffa5d None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:57.076493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:57.083704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1128/2215] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:57 2022] PUT /identity/v3/projects/60dd86f5ec50424c9426cbb039ffc1e5/users/083f1b8c7d5247ac8655bcf8b5564bf3/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:57.086046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadd951-509d-46f9-bf64-8a5c695cc5e8 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:57.086559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadd951-509d-46f9-bf64-8a5c695cc5e8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:57.086883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadd951-509d-46f9-bf64-8a5c695cc5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:57.097773 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632e203-b101-44e8-8ebc-4a25018ffa5d None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:57.128357 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-5dadd951-509d-46f9-bf64-8a5c695cc5e8 None None] MFA Rules not processed for user `083f1b8c7d5247ac8655bcf8b5564bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:57.129631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5dadd951-509d-46f9-bf64-8a5c695cc5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:57.218004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c85bab-a007-4fcf-aa5f-d527e8fa80c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:57.219407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c85bab-a007-4fcf-aa5f-d527e8fa80c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:57.219854 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c85bab-a007-4fcf-aa5f-d527e8fa80c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:57.220290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c85bab-a007-4fcf-aa5f-d527e8fa80c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:57.221333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c85bab-a007-4fcf-aa5f-d527e8fa80c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:57.223918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c85bab-a007-4fcf-aa5f-d527e8fa80c2 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:57.244094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1129/2216] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:57.258096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadd951-509d-46f9-bf64-8a5c695cc5e8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:57.267680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5dadd951-509d-46f9-bf64-8a5c695cc5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:57.341200 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadd951-509d-46f9-bf64-8a5c695cc5e8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:57.345360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e76976-b2af-4006-a9b8-afb47ec05aa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:57.346182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e76976-b2af-4006-a9b8-afb47ec05aa1 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:57.346317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e76976-b2af-4006-a9b8-afb47ec05aa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:57.346580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e76976-b2af-4006-a9b8-afb47ec05aa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:57.348500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-29e76976-b2af-4006-a9b8-afb47ec05aa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:57.418597 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1088/2217] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:02:57 2022] POST /identity/v3/projects => generated 381 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:57.428871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e76976-b2af-4006-a9b8-afb47ec05aa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:57.430428 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e76976-b2af-4006-a9b8-afb47ec05aa1 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:57.433116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c85bab-a007-4fcf-aa5f-d527e8fa80c2 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:57.440923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1130/2218] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:57.552735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c85bab-a007-4fcf-aa5f-d527e8fa80c2 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:57.563999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085a2e8-cbc7-4ac7-ac79-49db2b596bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:57.567501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085a2e8-cbc7-4ac7-ac79-49db2b596bda admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:57.567874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085a2e8-cbc7-4ac7-ac79-49db2b596bda admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:57.568159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085a2e8-cbc7-4ac7-ac79-49db2b596bda admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:57.569968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085a2e8-cbc7-4ac7-ac79-49db2b596bda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:57.576284 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085a2e8-cbc7-4ac7-ac79-49db2b596bda admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:57.683971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:02:57.693118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1089/2219] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:02:57 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:02:57.694765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e76976-b2af-4006-a9b8-afb47ec05aa1 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:57.715842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e76976-b2af-4006-a9b8-afb47ec05aa1 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:57.733923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9962b-4a60-4838-8502-5a1d02dddd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:57.736757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9962b-4a60-4838-8502-5a1d02dddd12 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:57.736757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9962b-4a60-4838-8502-5a1d02dddd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:57.737058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9962b-4a60-4838-8502-5a1d02dddd12 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:57.738900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9962b-4a60-4838-8502-5a1d02dddd12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:57.742258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9962b-4a60-4838-8502-5a1d02dddd12 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:57.778674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1131/2220] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:02:57.789883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085a2e8-cbc7-4ac7-ac79-49db2b596bda admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:57.819278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085a2e8-cbc7-4ac7-ac79-49db2b596bda admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:57.834534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc257b1-d39f-4dec-97d0-a4feaf533d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:57.834924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc257b1-d39f-4dec-97d0-a4feaf533d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:57.835759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc257b1-d39f-4dec-97d0-a4feaf533d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:57.837621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc257b1-d39f-4dec-97d0-a4feaf533d72 admin admin] PATH_INFO: `/v3/projects/58716aa6e08343a3892c75353ecf5a52/users/7a6403c078b5442896e6182ebfcb134b/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:57.909363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc257b1-d39f-4dec-97d0-a4feaf533d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58716aa6e08343a3892c75353ecf5a52, user_id=7a6403c078b5442896e6182ebfcb134b, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:57.918260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc257b1-d39f-4dec-97d0-a4feaf533d72 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:58.080519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1090/2221] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:57 2022] PUT /identity/v3/projects/58716aa6e08343a3892c75353ecf5a52/users/7a6403c078b5442896e6182ebfcb134b/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:58.084555 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9962b-4a60-4838-8502-5a1d02dddd12 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:58.109232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9962b-4a60-4838-8502-5a1d02dddd12 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:58.229532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067c1c-8df8-42c0-8841-ddc047b82d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:58.230377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067c1c-8df8-42c0-8841-ddc047b82d43 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:58.230791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067c1c-8df8-42c0-8841-ddc047b82d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:58.231166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067c1c-8df8-42c0-8841-ddc047b82d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:58.232625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40067c1c-8df8-42c0-8841-ddc047b82d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:58.234653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40067c1c-8df8-42c0-8841-ddc047b82d43 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:58.246910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1132/2222] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:02:58.250926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc257b1-d39f-4dec-97d0-a4feaf533d72 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:58.268359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc257b1-d39f-4dec-97d0-a4feaf533d72 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:58.303470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84186e3d-ce93-4ed9-a417-79bc679b0705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:58.304407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84186e3d-ce93-4ed9-a417-79bc679b0705 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:58.315770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84186e3d-ce93-4ed9-a417-79bc679b0705 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:58.316137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84186e3d-ce93-4ed9-a417-79bc679b0705 admin admin] PATH_INFO: `/v3/projects/58716aa6e08343a3892c75353ecf5a52/users/7a6403c078b5442896e6182ebfcb134b/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:58.329872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84186e3d-ce93-4ed9-a417-79bc679b0705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58716aa6e08343a3892c75353ecf5a52, user_id=7a6403c078b5442896e6182ebfcb134b, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:58.346120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84186e3d-ce93-4ed9-a417-79bc679b0705 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:58.375502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1091/2223] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:02:58 2022] PUT /identity/v3/projects/58716aa6e08343a3892c75353ecf5a52/users/7a6403c078b5442896e6182ebfcb134b/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:58.388697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463555-45ac-403f-8ba9-3a45b4080754 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:58.388697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463555-45ac-403f-8ba9-3a45b4080754 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:58.388697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463555-45ac-403f-8ba9-3a45b4080754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:58.445425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2e463555-45ac-403f-8ba9-3a45b4080754 None None] MFA Rules not processed for user `7a6403c078b5442896e6182ebfcb134b`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:02:58.447205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2e463555-45ac-403f-8ba9-3a45b4080754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:58.612546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1133/2224] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:02:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:02:58.626324 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84186e3d-ce93-4ed9-a417-79bc679b0705 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:58.739287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84186e3d-ce93-4ed9-a417-79bc679b0705 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:58.757373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e754a-5178-429c-9dc9-ac68355b96c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:58.757373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e754a-5178-429c-9dc9-ac68355b96c6 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:58.757373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e754a-5178-429c-9dc9-ac68355b96c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:58.757373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e754a-5178-429c-9dc9-ac68355b96c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:58.769979 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e754a-5178-429c-9dc9-ac68355b96c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:58.772620 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e754a-5178-429c-9dc9-ac68355b96c6 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:58.786082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1092/2225] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:02:59.147385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e463555-45ac-403f-8ba9-3a45b4080754 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:59.149893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2e463555-45ac-403f-8ba9-3a45b4080754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:59.292587 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e463555-45ac-403f-8ba9-3a45b4080754 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:59.301809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:59.302985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:59.303417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:59.303884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] PATH_INFO: `/v3/users/bc76fd7a5bf24013a2d382657a29d4de` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:59.306277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc76fd7a5bf24013a2d382657a29d4de)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:59.308624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:59.659478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc76fd7a5bf24013a2d382657a29d4de'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:59.690658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc76fd7a5bf24013a2d382657a29d4de'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:59.707293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc76fd7a5bf24013a2d382657a29d4de'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:59.713153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc76fd7a5bf24013a2d382657a29d4de'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:59.793045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e754a-5178-429c-9dc9-ac68355b96c6 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:02:59.795505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1a7e754a-5178-429c-9dc9-ac68355b96c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:59.807898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc76fd7a5bf24013a2d382657a29d4de'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:02:59.810372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e754a-5178-429c-9dc9-ac68355b96c6 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:02:59.815533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1134/2226] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:02:59 2022] DELETE /identity/v3/users/bc76fd7a5bf24013a2d382657a29d4de => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:02:59.816239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328d4cb-fc57-40ba-8d37-11c0dcf64e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:02:59.816938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328d4cb-fc57-40ba-8d37-11c0dcf64e5e service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:02:59.817401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328d4cb-fc57-40ba-8d37-11c0dcf64e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:02:59.817869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328d4cb-fc57-40ba-8d37-11c0dcf64e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:02:59.821140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8328d4cb-fc57-40ba-8d37-11c0dcf64e5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:02:59.962471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328d4cb-fc57-40ba-8d37-11c0dcf64e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:02:59.963957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328d4cb-fc57-40ba-8d37-11c0dcf64e5e service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:02:59.974975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1093/2227] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:02:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:03:00.012349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:00.014451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:00.143462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:00.204410 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdfc12-9414-4de8-89fc-41a0dd8c2f48 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:00.210696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6403c078b5442896e6182ebfcb134b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58716aa6e08343a3892c75353ecf5a52', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:00.211670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:00.212595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:00.213109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:00.214183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:00.214700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:00.216588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:00.238780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1135/2228] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:00.242912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328d4cb-fc57-40ba-8d37-11c0dcf64e5e service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:00.258441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328d4cb-fc57-40ba-8d37-11c0dcf64e5e service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:00.266691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b8562-e54f-475d-82aa-5b3643258b6b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6403c078b5442896e6182ebfcb134b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58716aa6e08343a3892c75353ecf5a52', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:00.266691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b8562-e54f-475d-82aa-5b3643258b6b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:00.266691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b8562-e54f-475d-82aa-5b3643258b6b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:00.267021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b8562-e54f-475d-82aa-5b3643258b6b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:00.267725 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b8562-e54f-475d-82aa-5b3643258b6b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:00.269739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b8562-e54f-475d-82aa-5b3643258b6b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:00.329265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:00.332216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:00.444103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e56104-25f8-46d8-b5e2-57b874398544 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:00.449266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:00.450172 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:00.450456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:00.450737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] PATH_INFO: `/v3/projects/557563d484864d6f870d40dafcca08e1` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:00.452072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=557563d484864d6f870d40dafcca08e1)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:00.454061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:00.472728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1094/2229] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:00 2022] POST /identity/v3/projects => generated 354 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:03:00.485782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b8562-e54f-475d-82aa-5b3643258b6b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:00.562559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b8562-e54f-475d-82aa-5b3643258b6b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:00.578472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9eb3d-07c0-4339-b492-ec7ab1d9cbd8 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6403c078b5442896e6182ebfcb134b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58716aa6e08343a3892c75353ecf5a52', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:00.578472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9eb3d-07c0-4339-b492-ec7ab1d9cbd8 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:00.578472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9eb3d-07c0-4339-b492-ec7ab1d9cbd8 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:00.578771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9eb3d-07c0-4339-b492-ec7ab1d9cbd8 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:00.578771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9eb3d-07c0-4339-b492-ec7ab1d9cbd8 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:00.579010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9eb3d-07c0-4339-b492-ec7ab1d9cbd8 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:00.580794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9eb3d-07c0-4339-b492-ec7ab1d9cbd8 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:00.588425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557563d484864d6f870d40dafcca08e1', 'request_id': 'req-03c96da3-1f79-4516-aa29-8d422fb74987'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:00.595240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557563d484864d6f870d40dafcca08e1', 'request_id': 'req-03c96da3-1f79-4516-aa29-8d422fb74987'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:00.605160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1095/2230] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:00.606123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557563d484864d6f870d40dafcca08e1', 'request_id': 'req-03c96da3-1f79-4516-aa29-8d422fb74987'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:00.662278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9eb3d-07c0-4339-b492-ec7ab1d9cbd8 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:00.682807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9eb3d-07c0-4339-b492-ec7ab1d9cbd8 tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:00.711248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] The token cache is being invalidate because project 557563d484864d6f870d40dafcca08e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:00.712227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:00.713155 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:00.716394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1136/2231] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:00 2022] DELETE /identity/v3/projects/557563d484864d6f870d40dafcca08e1 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:00.722194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:00.733277 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:00.819964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:00.827872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6403c078b5442896e6182ebfcb134b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58716aa6e08343a3892c75353ecf5a52', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:00.828898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:00.829195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:00.829479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] PATH_INFO: `/v3/projects/0a38052f91084af5acf67623fc04f5c7` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:00.832145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0a38052f91084af5acf67623fc04f5c7)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:00.834127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:00.951010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c96da3-1f79-4516-aa29-8d422fb74987 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:00.956186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:00.957368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:00.957858 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:00.958335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] PATH_INFO: `/v3/users/797d8023bf5043d08706f3335a177601` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:00.959718 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=797d8023bf5043d08706f3335a177601)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:00.962465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:01.019200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a38052f91084af5acf67623fc04f5c7', 'request_id': 'req-f46e4563-ece4-46b5-a7f3-bce2d76e880b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.027120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a38052f91084af5acf67623fc04f5c7', 'request_id': 'req-f46e4563-ece4-46b5-a7f3-bce2d76e880b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.113004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a38052f91084af5acf67623fc04f5c7', 'request_id': 'req-f46e4563-ece4-46b5-a7f3-bce2d76e880b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.133692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] The token cache is being invalidate because project 0a38052f91084af5acf67623fc04f5c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:01.135458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.136480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.141601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1096/2232] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:00 2022] DELETE /identity/v3/projects/0a38052f91084af5acf67623fc04f5c7 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:03:01.218697 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '797d8023bf5043d08706f3335a177601'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.235101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '797d8023bf5043d08706f3335a177601'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.244742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '797d8023bf5043d08706f3335a177601'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.246107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '797d8023bf5043d08706f3335a177601'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.499281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '797d8023bf5043d08706f3335a177601'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:01.506661 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1137/2233] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:00 2022] DELETE /identity/v3/users/797d8023bf5043d08706f3335a177601 => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:03:01.900293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:02.151942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:02.153627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:02.187943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:02.190188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:02.329534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e4563-ece4-46b5-a7f3-bce2d76e880b tempest-VolumeQuotaClassesTest-1613400182 tempest-VolumeQuotaClassesTest-1613400182-project-admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:02.337472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca2d80-bc1a-434f-8338-f65451bd8175 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:02.339270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:02.340260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:02.340554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:02.341029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] PATH_INFO: `/v3/users/b56a2b4385b7457a88d6c0cb907080df` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:02.342263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56a2b4385b7457a88d6c0cb907080df)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:02.344522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:02.346071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:02.347100 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:02.347400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:02.347758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] PATH_INFO: `/v3/projects/d7495b8c09f7452b9c50d9a2cf6048fb` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:02.349566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7495b8c09f7452b9c50d9a2cf6048fb)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:02.352008 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:02.564073 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7495b8c09f7452b9c50d9a2cf6048fb', 'request_id': 'req-29217507-ce96-4104-ad6e-8535bc481928'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.571039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56a2b4385b7457a88d6c0cb907080df'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.575412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7495b8c09f7452b9c50d9a2cf6048fb', 'request_id': 'req-29217507-ce96-4104-ad6e-8535bc481928'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.584082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56a2b4385b7457a88d6c0cb907080df'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.585343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56a2b4385b7457a88d6c0cb907080df'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.618624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7495b8c09f7452b9c50d9a2cf6048fb', 'request_id': 'req-29217507-ce96-4104-ad6e-8535bc481928'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.718679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56a2b4385b7457a88d6c0cb907080df'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.739778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56a2b4385b7457a88d6c0cb907080df'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.759523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1097/2234] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:02 2022] DELETE /identity/v3/users/b56a2b4385b7457a88d6c0cb907080df => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:02.760543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:02.762031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:02.814076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] The token cache is being invalidate because project d7495b8c09f7452b9c50d9a2cf6048fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:02.815521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.817242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-29217507-ce96-4104-ad6e-8535bc481928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:02.824570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1138/2235] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:02 2022] DELETE /identity/v3/projects/d7495b8c09f7452b9c50d9a2cf6048fb => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:02.826379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94ea6-cc60-4ce2-b361-143538a4aa0f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:02.827288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94ea6-cc60-4ce2-b361-143538a4aa0f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:02.827761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94ea6-cc60-4ce2-b361-143538a4aa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:02.874440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e15dcd-c644-4b19-83dc-b9ba5f92093c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:02.878591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-afb94ea6-cc60-4ce2-b361-143538a4aa0f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:02.884232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:02.884232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:02.884679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-afb94ea6-cc60-4ce2-b361-143538a4aa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:02.884799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:02.884799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:02.886258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:02.966105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1139/2236] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:02.969502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a2961-4a1c-4510-92b7-d236d2262a9f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:02.969988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a2961-4a1c-4510-92b7-d236d2262a9f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:02.970263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a2961-4a1c-4510-92b7-d236d2262a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:02.978184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:02.981215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:02.997048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1098/2237] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:02.997757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ba9a2961-4a1c-4510-92b7-d236d2262a9f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:02.998117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:02.999598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ba9a2961-4a1c-4510-92b7-d236d2262a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:02.999835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:03.010689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1140/2238] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:03.018575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f89354-afe4-4fde-a27e-8181a2aa3e98 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.018864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f89354-afe4-4fde-a27e-8181a2aa3e98 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.019167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f89354-afe4-4fde-a27e-8181a2aa3e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.044242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-68f89354-afe4-4fde-a27e-8181a2aa3e98 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:03.045618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-68f89354-afe4-4fde-a27e-8181a2aa3e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:03.058401 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1141/2239] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:03:03.059854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f89354-afe4-4fde-a27e-8181a2aa3e98 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:03.072823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f89354-afe4-4fde-a27e-8181a2aa3e98 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:03.077815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917fed3-bc40-4380-b17a-26deb8e85538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:03.078497 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917fed3-bc40-4380-b17a-26deb8e85538 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.078962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917fed3-bc40-4380-b17a-26deb8e85538 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.079249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917fed3-bc40-4380-b17a-26deb8e85538 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.079999 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917fed3-bc40-4380-b17a-26deb8e85538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:03.080326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917fed3-bc40-4380-b17a-26deb8e85538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:03.082294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917fed3-bc40-4380-b17a-26deb8e85538 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:03.091550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14976a-5ff2-4090-9c91-9631beb841ed service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:03.114345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8c707-f17b-409d-889a-77a0d53c0597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:03.114943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8c707-f17b-409d-889a-77a0d53c0597 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.115284 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8c707-f17b-409d-889a-77a0d53c0597 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.115729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1142/2240] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:03.126696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8c707-f17b-409d-889a-77a0d53c0597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.128558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:03.130377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917fed3-bc40-4380-b17a-26deb8e85538 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:03.147416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917fed3-bc40-4380-b17a-26deb8e85538 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:03.150547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8c707-f17b-409d-889a-77a0d53c0597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:03.152181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8c707-f17b-409d-889a-77a0d53c0597 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:03.156440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a74d3c-72cc-484f-81c5-237be0814343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:03.157402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74d3c-72cc-484f-81c5-237be0814343 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.157837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74d3c-72cc-484f-81c5-237be0814343 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.158282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74d3c-72cc-484f-81c5-237be0814343 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.159130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a74d3c-72cc-484f-81c5-237be0814343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:03.162804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a74d3c-72cc-484f-81c5-237be0814343 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:03.178390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1099/2241] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:03.311540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1143/2242] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:03 2022] POST /identity/v3/projects => generated 389 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:03.316008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8c707-f17b-409d-889a-77a0d53c0597 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:03.329783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8c707-f17b-409d-889a-77a0d53c0597 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:03.494152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0036a-a97f-4361-8cdb-b9d912face82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:03.495031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0036a-a97f-4361-8cdb-b9d912face82 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.495393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0036a-a97f-4361-8cdb-b9d912face82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.495747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0036a-a97f-4361-8cdb-b9d912face82 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.496546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0036a-a97f-4361-8cdb-b9d912face82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:03.499818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0036a-a97f-4361-8cdb-b9d912face82 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:03.604757 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1100/2243] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:03 2022] POST /identity/v3/users => generated 295 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:03.609805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a74d3c-72cc-484f-81c5-237be0814343 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:03.622484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a74d3c-72cc-484f-81c5-237be0814343 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:03.628513 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b3386-e2c5-4962-95e2-d189ae9be22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:03.629488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b3386-e2c5-4962-95e2-d189ae9be22f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.629986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b3386-e2c5-4962-95e2-d189ae9be22f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.630453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b3386-e2c5-4962-95e2-d189ae9be22f admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.631418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b3386-e2c5-4962-95e2-d189ae9be22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:03.633557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b3386-e2c5-4962-95e2-d189ae9be22f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:03.655617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1144/2244] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:03.659781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0036a-a97f-4361-8cdb-b9d912face82 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:03.681585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0036a-a97f-4361-8cdb-b9d912face82 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:03.688061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca321e4-6d00-47f3-a9f1-0a99e707d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:03.688840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca321e4-6d00-47f3-a9f1-0a99e707d739 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.689133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca321e4-6d00-47f3-a9f1-0a99e707d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.689411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca321e4-6d00-47f3-a9f1-0a99e707d739 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.690165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca321e4-6d00-47f3-a9f1-0a99e707d739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:03.691847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca321e4-6d00-47f3-a9f1-0a99e707d739 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:03.706001 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1101/2245] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:03.710309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b3386-e2c5-4962-95e2-d189ae9be22f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:03.721879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b3386-e2c5-4962-95e2-d189ae9be22f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:03.726803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963a7fe-a48c-40c5-bbd3-7c202b0ab426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:03.728799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963a7fe-a48c-40c5-bbd3-7c202b0ab426 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.729093 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963a7fe-a48c-40c5-bbd3-7c202b0ab426 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.729373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963a7fe-a48c-40c5-bbd3-7c202b0ab426 admin admin] PATH_INFO: `/v3/projects/3d603cf539d940d39df533ae9dece453/users/fde9403ad838416abc7cdab4d9405362/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.749637 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963a7fe-a48c-40c5-bbd3-7c202b0ab426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d603cf539d940d39df533ae9dece453, user_id=fde9403ad838416abc7cdab4d9405362, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:03.752520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963a7fe-a48c-40c5-bbd3-7c202b0ab426 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:03.783862 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca321e4-6d00-47f3-a9f1-0a99e707d739 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:03.785846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-cca321e4-6d00-47f3-a9f1-0a99e707d739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:03.801222 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca321e4-6d00-47f3-a9f1-0a99e707d739 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:03.809668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:03.810606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.810897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.811183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] PATH_INFO: `/v3/users/d81832021ecb4cf983fe91f979d8321a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.812518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81832021ecb4cf983fe91f979d8321a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:03.814982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:03.816613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1145/2246] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:03 2022] PUT /identity/v3/projects/3d603cf539d940d39df533ae9dece453/users/fde9403ad838416abc7cdab4d9405362/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:03.829646 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae074cb0-bfd5-452b-8f42-2a90a35c33bd None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:03.832371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae074cb0-bfd5-452b-8f42-2a90a35c33bd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:03.832371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae074cb0-bfd5-452b-8f42-2a90a35c33bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:03.865742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ae074cb0-bfd5-452b-8f42-2a90a35c33bd None None] MFA Rules not processed for user `fde9403ad838416abc7cdab4d9405362`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:03.867074 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ae074cb0-bfd5-452b-8f42-2a90a35c33bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:03.889861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:03.943014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1146/2247] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:03.953674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae074cb0-bfd5-452b-8f42-2a90a35c33bd None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:03.955066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ae074cb0-bfd5-452b-8f42-2a90a35c33bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:04.023125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae074cb0-bfd5-452b-8f42-2a90a35c33bd None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:04.028085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d2f19-0538-44da-866d-568121160e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:04.029766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d2f19-0538-44da-866d-568121160e7a service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:04.030215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d2f19-0538-44da-866d-568121160e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:04.031670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d2f19-0538-44da-866d-568121160e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:04.044934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d2f19-0538-44da-866d-568121160e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:04.046826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d2f19-0538-44da-866d-568121160e7a service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:04.047423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81832021ecb4cf983fe91f979d8321a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.058602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81832021ecb4cf983fe91f979d8321a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.059415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81832021ecb4cf983fe91f979d8321a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.060156 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1147/2248] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:03:04.061096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d2f19-0538-44da-866d-568121160e7a service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:04.068226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5c1d2f19-0538-44da-866d-568121160e7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:04.167652 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81832021ecb4cf983fe91f979d8321a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.201236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36a235e0-9abe-4af0-a7f1-bf910c18316e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81832021ecb4cf983fe91f979d8321a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.216868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1102/2249] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:03 2022] DELETE /identity/v3/users/d81832021ecb4cf983fe91f979d8321a => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:03:04.221210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb0379-a779-4547-a81c-8a71f92eceb2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:04.222974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb0379-a779-4547-a81c-8a71f92eceb2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:04.223601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb0379-a779-4547-a81c-8a71f92eceb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:04.257106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c5fb0379-a779-4547-a81c-8a71f92eceb2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:04.258467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c5fb0379-a779-4547-a81c-8a71f92eceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:04.267767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d2f19-0538-44da-866d-568121160e7a service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:04.349595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1103/2250] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:04.372900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb0379-a779-4547-a81c-8a71f92eceb2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:04.374444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c5fb0379-a779-4547-a81c-8a71f92eceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:04.397536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:04.398570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:04.398870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:04.399159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] PATH_INFO: `/v3/projects/01fbbc85cfbe461ca8ce7685281154b6` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:04.417202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=01fbbc85cfbe461ca8ce7685281154b6)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:04.422900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:04.530527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb0379-a779-4547-a81c-8a71f92eceb2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:04.536228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d325a86-52f5-40e9-b04f-7e5df026c2e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:04.537069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d325a86-52f5-40e9-b04f-7e5df026c2e8 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:04.537658 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d325a86-52f5-40e9-b04f-7e5df026c2e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:04.545201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d325a86-52f5-40e9-b04f-7e5df026c2e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:04.572158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d325a86-52f5-40e9-b04f-7e5df026c2e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:04.573691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d325a86-52f5-40e9-b04f-7e5df026c2e8 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:04.603637 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1104/2251] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:04.617453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01fbbc85cfbe461ca8ce7685281154b6', 'request_id': 'req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.637394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01fbbc85cfbe461ca8ce7685281154b6', 'request_id': 'req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.647632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01fbbc85cfbe461ca8ce7685281154b6', 'request_id': 'req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.802129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] The token cache is being invalidate because project 01fbbc85cfbe461ca8ce7685281154b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:04.802572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.803575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-0f1fa807-ae4d-45a0-84d5-5cd22300c1ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:04.847110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1148/2252] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:04 2022] DELETE /identity/v3/projects/01fbbc85cfbe461ca8ce7685281154b6 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:04.852282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d325a86-52f5-40e9-b04f-7e5df026c2e8 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:04.853878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1d325a86-52f5-40e9-b04f-7e5df026c2e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:04.877235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d325a86-52f5-40e9-b04f-7e5df026c2e8 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:04.950229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:04.951418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:04.951930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:04.952391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] PATH_INFO: `/v3/users/7a6403c078b5442896e6182ebfcb134b` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:04.954167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6403c078b5442896e6182ebfcb134b)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:04.957406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:05.129872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e1329-a556-4e58-b329-05c8f23e9234 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:05.130300 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e1329-a556-4e58-b329-05c8f23e9234 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:05.130651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e1329-a556-4e58-b329-05c8f23e9234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:05.163656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-344e1329-a556-4e58-b329-05c8f23e9234 None None] MFA Rules not processed for user `019c27e5d7564e82a2c666cbc87c378e`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:05.166961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-344e1329-a556-4e58-b329-05c8f23e9234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:05.250408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6403c078b5442896e6182ebfcb134b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:05.265342 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6403c078b5442896e6182ebfcb134b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:05.270873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6403c078b5442896e6182ebfcb134b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:05.278837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1149/2253] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:05.290691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e1329-a556-4e58-b329-05c8f23e9234 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:05.293283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-344e1329-a556-4e58-b329-05c8f23e9234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:05.383596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e1329-a556-4e58-b329-05c8f23e9234 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:05.388836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfefe1-5381-45f9-a170-9bb7deffae35 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca33ef49ba44832a7fb45b2317d211c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:05.389528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfefe1-5381-45f9-a170-9bb7deffae35 service glance] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:05.389822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfefe1-5381-45f9-a170-9bb7deffae35 service glance] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:05.390910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfefe1-5381-45f9-a170-9bb7deffae35 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:05.394330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4cdfefe1-5381-45f9-a170-9bb7deffae35 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:05.402149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6403c078b5442896e6182ebfcb134b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:05.430842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6403c078b5442896e6182ebfcb134b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:05.436766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:05.450184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1105/2254] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:04 2022] DELETE /identity/v3/users/7a6403c078b5442896e6182ebfcb134b => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:05.461559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:05.488350 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:05.489049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfefe1-5381-45f9-a170-9bb7deffae35 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:05.490948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfefe1-5381-45f9-a170-9bb7deffae35 service glance] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:05.503388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1150/2255] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:03:05.670292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e3e36-87c8-488a-aaf6-957ce7da024f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:05.676590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:05.677709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:05.678119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:05.678668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] PATH_INFO: `/v3/projects/2d91b39df422425d8e966ff1d23149a9` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:05.680686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d91b39df422425d8e966ff1d23149a9)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:05.683300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:05.895148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d91b39df422425d8e966ff1d23149a9', 'request_id': 'req-dbf0797c-4dc1-4895-9d2c-210c19568f8b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:05.896752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2699c-5dd0-44eb-8777-913722d47c11 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:05.897613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2699c-5dd0-44eb-8777-913722d47c11 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:05.898040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2699c-5dd0-44eb-8777-913722d47c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:05.911990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d91b39df422425d8e966ff1d23149a9', 'request_id': 'req-dbf0797c-4dc1-4895-9d2c-210c19568f8b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:05.940912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-7cd2699c-5dd0-44eb-8777-913722d47c11 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:05.942469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7cd2699c-5dd0-44eb-8777-913722d47c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:06.002510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d91b39df422425d8e966ff1d23149a9', 'request_id': 'req-dbf0797c-4dc1-4895-9d2c-210c19568f8b'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:06.029140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] The token cache is being invalidate because project 2d91b39df422425d8e966ff1d23149a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:06.029522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:06.030406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-dbf0797c-4dc1-4895-9d2c-210c19568f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:06.032223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1106/2256] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:05 2022] DELETE /identity/v3/projects/2d91b39df422425d8e966ff1d23149a9 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:06.033193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1151/2257] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:03:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:06.043220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f210e63-2b1e-4950-be6c-71da2cc0eae4 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:06.043621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f210e63-2b1e-4950-be6c-71da2cc0eae4 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:06.044120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f210e63-2b1e-4950-be6c-71da2cc0eae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:06.116015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-4f210e63-2b1e-4950-be6c-71da2cc0eae4 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:06.120727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2699c-5dd0-44eb-8777-913722d47c11 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:06.120915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4f210e63-2b1e-4950-be6c-71da2cc0eae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:06.121002 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7cd2699c-5dd0-44eb-8777-913722d47c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:06.248804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1107/2258] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:06.248804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f210e63-2b1e-4950-be6c-71da2cc0eae4 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:06.250950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4f210e63-2b1e-4950-be6c-71da2cc0eae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:06.270032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f210e63-2b1e-4950-be6c-71da2cc0eae4 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:06.278686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:06.279631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:06.279924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:06.281454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] PATH_INFO: `/v3/projects/58716aa6e08343a3892c75353ecf5a52` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:06.282818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58716aa6e08343a3892c75353ecf5a52)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:06.285477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:06.296857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2699c-5dd0-44eb-8777-913722d47c11 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:06.306788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e162258-3e3f-4d73-aa9b-b4b8bea6c091 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:06.308934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e162258-3e3f-4d73-aa9b-b4b8bea6c091 service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:06.309450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e162258-3e3f-4d73-aa9b-b4b8bea6c091 service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:06.309879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e162258-3e3f-4d73-aa9b-b4b8bea6c091 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:06.312101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7e162258-3e3f-4d73-aa9b-b4b8bea6c091 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:06.336502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e162258-3e3f-4d73-aa9b-b4b8bea6c091 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:06.338370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e162258-3e3f-4d73-aa9b-b4b8bea6c091 service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:06.347596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1152/2259] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:03:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 24 20:03:06.349372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beb795-4082-48d1-9ae0-9197e04d3a0f None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:06.349651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beb795-4082-48d1-9ae0-9197e04d3a0f None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:06.350171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beb795-4082-48d1-9ae0-9197e04d3a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:06.396490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c0beb795-4082-48d1-9ae0-9197e04d3a0f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:06.401445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c0beb795-4082-48d1-9ae0-9197e04d3a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:06.433791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1153/2260] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:06.439065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0beb795-4082-48d1-9ae0-9197e04d3a0f None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:06.470496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58716aa6e08343a3892c75353ecf5a52', 'request_id': 'req-85993172-922a-483a-ba50-c12b7c5d7832'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:06.474831 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0beb795-4082-48d1-9ae0-9197e04d3a0f None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:06.486560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58716aa6e08343a3892c75353ecf5a52', 'request_id': 'req-85993172-922a-483a-ba50-c12b7c5d7832'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:06.573710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58716aa6e08343a3892c75353ecf5a52', 'request_id': 'req-85993172-922a-483a-ba50-c12b7c5d7832'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:06.598201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] The token cache is being invalidate because project 58716aa6e08343a3892c75353ecf5a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:06.598682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:06.601495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-85993172-922a-483a-ba50-c12b7c5d7832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:06.609605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1108/2261] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:06 2022] DELETE /identity/v3/projects/58716aa6e08343a3892c75353ecf5a52 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:06.622676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c35c9d-db50-4d20-9686-d3e8e291f45e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:06.623790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:06.631732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c35c9d-db50-4d20-9686-d3e8e291f45e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:06.632257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c35c9d-db50-4d20-9686-d3e8e291f45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:06.660127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:06.665994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:06.666577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:06.667706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:06.668726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:06.672141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:06.678864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c2c35c9d-db50-4d20-9686-d3e8e291f45e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:06.684619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:06.695175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c2c35c9d-db50-4d20-9686-d3e8e291f45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:06.714421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1109/2262] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:06.722373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fd111-68bb-435f-90c8-2895dc63c959 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:06.723031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fd111-68bb-435f-90c8-2895dc63c959 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:06.723464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fd111-68bb-435f-90c8-2895dc63c959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:06.757059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1154/2263] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:06.762799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:06.768990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:06.785291 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8d3fd111-68bb-435f-90c8-2895dc63c959 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:06.793869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8d3fd111-68bb-435f-90c8-2895dc63c959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:06.823392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aaf6c-89da-4ee3-9ee4-70730a0bdea8 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:06.825788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1110/2264] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:03:06.834616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fd111-68bb-435f-90c8-2895dc63c959 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:06.851296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457e264-43d7-485f-92d8-5c5ef18de987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:06.852173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e264-43d7-485f-92d8-5c5ef18de987 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:06.853211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e264-43d7-485f-92d8-5c5ef18de987 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:06.853638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e264-43d7-485f-92d8-5c5ef18de987 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:06.861019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457e264-43d7-485f-92d8-5c5ef18de987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:06.865318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457e264-43d7-485f-92d8-5c5ef18de987 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:06.869447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fd111-68bb-435f-90c8-2895dc63c959 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:06.883232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcedbf-23bd-4353-90ca-8cedd862eb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:06.893722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcedbf-23bd-4353-90ca-8cedd862eb11 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:06.894080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcedbf-23bd-4353-90ca-8cedd862eb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:06.894361 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcedbf-23bd-4353-90ca-8cedd862eb11 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:06.899201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcedbf-23bd-4353-90ca-8cedd862eb11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:06.906522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcedbf-23bd-4353-90ca-8cedd862eb11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:06.910347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcedbf-23bd-4353-90ca-8cedd862eb11 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:06.965851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1111/2265] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:06.969466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcedbf-23bd-4353-90ca-8cedd862eb11 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:06.986002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcedbf-23bd-4353-90ca-8cedd862eb11 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:06.997859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c429048-c0d5-43ab-84ed-d6689e70c602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:07.008395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c429048-c0d5-43ab-84ed-d6689e70c602 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:07.008395 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c429048-c0d5-43ab-84ed-d6689e70c602 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:07.009490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c429048-c0d5-43ab-84ed-d6689e70c602 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:07.018179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c429048-c0d5-43ab-84ed-d6689e70c602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:07.023596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c429048-c0d5-43ab-84ed-d6689e70c602 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:07.091857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1155/2266] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:06 2022] POST /identity/v3/projects => generated 359 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:07.093508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457e264-43d7-485f-92d8-5c5ef18de987 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:07.185277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1112/2267] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:06 2022] POST /identity/v3/projects => generated 393 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:03:07.187022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c429048-c0d5-43ab-84ed-d6689e70c602 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:07.249226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457e264-43d7-485f-92d8-5c5ef18de987 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:07.494578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06232c-eab2-4844-ba4e-9cb38a1a3424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:07.494578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06232c-eab2-4844-ba4e-9cb38a1a3424 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:07.494578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06232c-eab2-4844-ba4e-9cb38a1a3424 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:07.494578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06232c-eab2-4844-ba4e-9cb38a1a3424 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:07.496517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06232c-eab2-4844-ba4e-9cb38a1a3424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:07.496517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06232c-eab2-4844-ba4e-9cb38a1a3424 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:07.496796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c429048-c0d5-43ab-84ed-d6689e70c602 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:07.496796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01966de0-3f3c-47df-8508-aec6895d9487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:07.496796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01966de0-3f3c-47df-8508-aec6895d9487 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:07.496796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01966de0-3f3c-47df-8508-aec6895d9487 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:07.510915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01966de0-3f3c-47df-8508-aec6895d9487 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:07.510915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01966de0-3f3c-47df-8508-aec6895d9487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:07.510915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01966de0-3f3c-47df-8508-aec6895d9487 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:07.859594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1113/2268] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:07 2022] POST /identity/v3/users => generated 297 bytes in 668 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:07.859699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1156/2269] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:07 2022] POST /identity/v3/users => generated 280 bytes in 763 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:07.859781 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01966de0-3f3c-47df-8508-aec6895d9487 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:07.861296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06232c-eab2-4844-ba4e-9cb38a1a3424 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:07.874313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06232c-eab2-4844-ba4e-9cb38a1a3424 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:07.882104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc5b6d-1252-4a31-9774-ccf4f7b625c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:07.883226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc5b6d-1252-4a31-9774-ccf4f7b625c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:07.883764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc5b6d-1252-4a31-9774-ccf4f7b625c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:07.884689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01966de0-3f3c-47df-8508-aec6895d9487 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:07.885071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc5b6d-1252-4a31-9774-ccf4f7b625c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:07.886488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc5b6d-1252-4a31-9774-ccf4f7b625c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:07.888831 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc5b6d-1252-4a31-9774-ccf4f7b625c7 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:07.899823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69402-fe16-4977-a961-cf1a0d0bca4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:07.900515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69402-fe16-4977-a961-cf1a0d0bca4f admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:07.900830 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69402-fe16-4977-a961-cf1a0d0bca4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:07.901472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69402-fe16-4977-a961-cf1a0d0bca4f admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:07.902390 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e69402-fe16-4977-a961-cf1a0d0bca4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:07.910951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1157/2270] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:07.912945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e69402-fe16-4977-a961-cf1a0d0bca4f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:07.914444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc5b6d-1252-4a31-9774-ccf4f7b625c7 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:07.973755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1114/2271] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:07.974732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69402-fe16-4977-a961-cf1a0d0bca4f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:07.979493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc5b6d-1252-4a31-9774-ccf4f7b625c7 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:07.996057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211722f-39e3-4953-bdbd-9f1a6e624e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:07.996798 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211722f-39e3-4953-bdbd-9f1a6e624e69 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:07.997094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211722f-39e3-4953-bdbd-9f1a6e624e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:07.997375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211722f-39e3-4953-bdbd-9f1a6e624e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:08.000768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211722f-39e3-4953-bdbd-9f1a6e624e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:08.002414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211722f-39e3-4953-bdbd-9f1a6e624e69 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:08.006811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69402-fe16-4977-a961-cf1a0d0bca4f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:08.016605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d902889-c6d7-45c3-aad8-397e913101df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:08.017327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d902889-c6d7-45c3-aad8-397e913101df admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:08.017612 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d902889-c6d7-45c3-aad8-397e913101df admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:08.017893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d902889-c6d7-45c3-aad8-397e913101df admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:08.018635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d902889-c6d7-45c3-aad8-397e913101df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:08.020259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d902889-c6d7-45c3-aad8-397e913101df admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:08.035197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1158/2272] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:08.037163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211722f-39e3-4953-bdbd-9f1a6e624e69 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:08.049414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1115/2273] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:08.050874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d902889-c6d7-45c3-aad8-397e913101df admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:08.052053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211722f-39e3-4953-bdbd-9f1a6e624e69 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:08.057600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2ed2d-9dda-4553-b3bb-e74136072a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:08.060403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2ed2d-9dda-4553-b3bb-e74136072a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:08.060403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2ed2d-9dda-4553-b3bb-e74136072a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:08.060403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2ed2d-9dda-4553-b3bb-e74136072a90 admin admin] PATH_INFO: `/v3/projects/6f35a475c4cc4cbcb6e4ca00bf31eab8/users/71624d33d7dc46bf92d0b851d3de6fac/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:08.072386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d902889-c6d7-45c3-aad8-397e913101df admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:08.079164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa8811-4ee1-4331-af40-47b19aa74eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:08.080481 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2ed2d-9dda-4553-b3bb-e74136072a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f35a475c4cc4cbcb6e4ca00bf31eab8, user_id=71624d33d7dc46bf92d0b851d3de6fac, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:08.080606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa8811-4ee1-4331-af40-47b19aa74eef admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:08.081381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa8811-4ee1-4331-af40-47b19aa74eef admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:08.081675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa8811-4ee1-4331-af40-47b19aa74eef admin admin] PATH_INFO: `/v3/projects/c6106147a3224d148e36e1c1a8b8d6e5/users/7288c84d23434997acbf3f0af85a9f99/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:08.083631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2ed2d-9dda-4553-b3bb-e74136072a90 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:08.101195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa8811-4ee1-4331-af40-47b19aa74eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6106147a3224d148e36e1c1a8b8d6e5, user_id=7288c84d23434997acbf3f0af85a9f99, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:08.103984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa8811-4ee1-4331-af40-47b19aa74eef admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:08.420533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1116/2274] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:08 2022] PUT /identity/v3/projects/c6106147a3224d148e36e1c1a8b8d6e5/users/7288c84d23434997acbf3f0af85a9f99/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:08.421660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1159/2275] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:08 2022] PUT /identity/v3/projects/6f35a475c4cc4cbcb6e4ca00bf31eab8/users/71624d33d7dc46bf92d0b851d3de6fac/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:03:08.438955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986deb-f012-4361-801a-4f376a81bc9b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:08.439616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986deb-f012-4361-801a-4f376a81bc9b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:08.440124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986deb-f012-4361-801a-4f376a81bc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:08.445326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4740-af6a-40ed-a255-52df9a725854 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:08.445814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4740-af6a-40ed-a255-52df9a725854 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:08.446850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4740-af6a-40ed-a255-52df9a725854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:08.502556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-63986deb-f012-4361-801a-4f376a81bc9b None None] MFA Rules not processed for user `7288c84d23434997acbf3f0af85a9f99`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:08.503824 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-63986deb-f012-4361-801a-4f376a81bc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:08.513714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-72ea4740-af6a-40ed-a255-52df9a725854 None None] MFA Rules not processed for user `71624d33d7dc46bf92d0b851d3de6fac`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:08.516405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-72ea4740-af6a-40ed-a255-52df9a725854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:08.648026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1117/2276] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:08.653815 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63986deb-f012-4361-801a-4f376a81bc9b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:08.702821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1160/2277] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:08 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:08.735976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4740-af6a-40ed-a255-52df9a725854 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:08.739205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-72ea4740-af6a-40ed-a255-52df9a725854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:08.813836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63986deb-f012-4361-801a-4f376a81bc9b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:08.825335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324db124-03d1-444c-8393-993df3a283d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:08.825335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324db124-03d1-444c-8393-993df3a283d0 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:08.825335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324db124-03d1-444c-8393-993df3a283d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:08.825335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324db124-03d1-444c-8393-993df3a283d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:08.830897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324db124-03d1-444c-8393-993df3a283d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:08.833073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324db124-03d1-444c-8393-993df3a283d0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:08.867291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4740-af6a-40ed-a255-52df9a725854 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:08.872262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84c587-98dd-4447-9c91-5d8a11d480b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:08.873338 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84c587-98dd-4447-9c91-5d8a11d480b1 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:08.874014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84c587-98dd-4447-9c91-5d8a11d480b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:08.874511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84c587-98dd-4447-9c91-5d8a11d480b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:08.885893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84c587-98dd-4447-9c91-5d8a11d480b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:08.890450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84c587-98dd-4447-9c91-5d8a11d480b1 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:08.923720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1161/2278] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:08 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:03:08.934799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84c587-98dd-4447-9c91-5d8a11d480b1 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:08.959796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84c587-98dd-4447-9c91-5d8a11d480b1 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:08.966314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1118/2279] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:08 2022] POST /identity/v3/projects => generated 391 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:08.971303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324db124-03d1-444c-8393-993df3a283d0 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:08.980763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:08.981741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:08.982053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:08.982375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] PATH_INFO: `/v3/users/71624d33d7dc46bf92d0b851d3de6fac` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:08.983818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] RBAC: Authorizing `identity:delete_user(user_id=71624d33d7dc46bf92d0b851d3de6fac)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:08.986254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:09.042945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:09.186588 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324db124-03d1-444c-8393-993df3a283d0 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:09.213006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b1396-a269-4a57-b653-9e2d315cd3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:09.213006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b1396-a269-4a57-b653-9e2d315cd3fa admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:09.213006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b1396-a269-4a57-b653-9e2d315cd3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:09.213006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b1396-a269-4a57-b653-9e2d315cd3fa admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:09.213711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b1396-a269-4a57-b653-9e2d315cd3fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:09.221185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b1396-a269-4a57-b653-9e2d315cd3fa admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:09.626440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71624d33d7dc46bf92d0b851d3de6fac'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:09.643205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71624d33d7dc46bf92d0b851d3de6fac'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:09.657586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71624d33d7dc46bf92d0b851d3de6fac'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:09.657586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71624d33d7dc46bf92d0b851d3de6fac'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:09.661002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1119/2280] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:08 2022] POST /identity/v3/users => generated 302 bytes in 690 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:09.664607 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b1396-a269-4a57-b653-9e2d315cd3fa admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:09.675384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b1396-a269-4a57-b653-9e2d315cd3fa admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:09.681347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cbe8b-019d-4f4e-8e51-ca20eb77a68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:09.682337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cbe8b-019d-4f4e-8e51-ca20eb77a68a admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:09.682831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cbe8b-019d-4f4e-8e51-ca20eb77a68a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:09.683304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cbe8b-019d-4f4e-8e51-ca20eb77a68a admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:09.684236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cbe8b-019d-4f4e-8e51-ca20eb77a68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:09.686075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cbe8b-019d-4f4e-8e51-ca20eb77a68a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:09.700267 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1120/2281] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:03:09.879215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cbe8b-019d-4f4e-8e51-ca20eb77a68a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:09.904632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cbe8b-019d-4f4e-8e51-ca20eb77a68a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:09.904632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21ca06-eafb-4869-a843-50b94e89fe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:09.904632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21ca06-eafb-4869-a843-50b94e89fe61 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:09.904632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21ca06-eafb-4869-a843-50b94e89fe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:09.905113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21ca06-eafb-4869-a843-50b94e89fe61 admin admin] PATH_INFO: `/v3/projects/1eec27259eb148c6b95a9844af6ede1a/users/14bfe5bc92814dcc983da69ae6bcfb7e/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:09.924553 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21ca06-eafb-4869-a843-50b94e89fe61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eec27259eb148c6b95a9844af6ede1a, user_id=14bfe5bc92814dcc983da69ae6bcfb7e, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:09.927647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21ca06-eafb-4869-a843-50b94e89fe61 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:10.088623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f4a41ef6-d088-4384-96d6-8ab177f6e64c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71624d33d7dc46bf92d0b851d3de6fac'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:10.096302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1162/2282] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:08 2022] DELETE /identity/v3/users/71624d33d7dc46bf92d0b851d3de6fac => generated 0 bytes in 1162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:10.100780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d7df2-1894-4afe-b873-df5d27ce383b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:10.101403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d7df2-1894-4afe-b873-df5d27ce383b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:10.101849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d7df2-1894-4afe-b873-df5d27ce383b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:10.133552 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1121/2283] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:09 2022] PUT /identity/v3/projects/1eec27259eb148c6b95a9844af6ede1a/users/14bfe5bc92814dcc983da69ae6bcfb7e/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:10.136828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21ca06-eafb-4869-a843-50b94e89fe61 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:10.138632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7c21ca06-eafb-4869-a843-50b94e89fe61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:10.144337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-af2d7df2-1894-4afe-b873-df5d27ce383b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:10.146651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-af2d7df2-1894-4afe-b873-df5d27ce383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:10.303733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21ca06-eafb-4869-a843-50b94e89fe61 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:10.320759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1163/2284] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:03:10.332909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d7df2-1894-4afe-b873-df5d27ce383b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:10.336181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-af2d7df2-1894-4afe-b873-df5d27ce383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:10.342657 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17636759-cd10-43b7-a628-6888297f63f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:10.344576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17636759-cd10-43b7-a628-6888297f63f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:10.345085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17636759-cd10-43b7-a628-6888297f63f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:10.346839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17636759-cd10-43b7-a628-6888297f63f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:10.347592 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17636759-cd10-43b7-a628-6888297f63f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:10.355717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17636759-cd10-43b7-a628-6888297f63f5 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:10.461037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1122/2285] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:10.465255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17636759-cd10-43b7-a628-6888297f63f5 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:10.483772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d7df2-1894-4afe-b873-df5d27ce383b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:10.489983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17636759-cd10-43b7-a628-6888297f63f5 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:10.494743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483d462-be06-47a5-95b3-8569e06f74ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:10.495422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483d462-be06-47a5-95b3-8569e06f74ab service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:10.495718 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483d462-be06-47a5-95b3-8569e06f74ab service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:10.496005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483d462-be06-47a5-95b3-8569e06f74ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:10.497210 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d724ab2-6bad-4df2-82f0-8913c458416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:10.503043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724ab2-6bad-4df2-82f0-8913c458416b admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:10.503344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724ab2-6bad-4df2-82f0-8913c458416b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:10.504134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724ab2-6bad-4df2-82f0-8913c458416b admin admin] PATH_INFO: `/v3/projects/1eec27259eb148c6b95a9844af6ede1a/users/14bfe5bc92814dcc983da69ae6bcfb7e/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:10.510103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d724ab2-6bad-4df2-82f0-8913c458416b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eec27259eb148c6b95a9844af6ede1a, user_id=14bfe5bc92814dcc983da69ae6bcfb7e, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:10.512836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d724ab2-6bad-4df2-82f0-8913c458416b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:10.526350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483d462-be06-47a5-95b3-8569e06f74ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:10.530668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483d462-be06-47a5-95b3-8569e06f74ab service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:10.544878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1123/2286] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:10 2022] PUT /identity/v3/projects/1eec27259eb148c6b95a9844af6ede1a/users/14bfe5bc92814dcc983da69ae6bcfb7e/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:10.557478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1164/2287] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:10.563428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e58b24-d336-41f6-87d1-0361d0ae7569 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:10.563985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e58b24-d336-41f6-87d1-0361d0ae7569 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:10.564460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e58b24-d336-41f6-87d1-0361d0ae7569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:10.659670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c2e58b24-d336-41f6-87d1-0361d0ae7569 None None] MFA Rules not processed for user `14bfe5bc92814dcc983da69ae6bcfb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:10.668053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c2e58b24-d336-41f6-87d1-0361d0ae7569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:10.792854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483d462-be06-47a5-95b3-8569e06f74ab service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:10.795073 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9483d462-be06-47a5-95b3-8569e06f74ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:10.819973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483d462-be06-47a5-95b3-8569e06f74ab service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:10.847420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1124/2288] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:10.861647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e58b24-d336-41f6-87d1-0361d0ae7569 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:10.864399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c2e58b24-d336-41f6-87d1-0361d0ae7569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:10.976839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:10.978751 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:10.979328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:10.980063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] PATH_INFO: `/v3/users/3174a496203248f38d55e640bcc2cf5b` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:10.981946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3174a496203248f38d55e640bcc2cf5b)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:10.984590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:11.013228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e58b24-d336-41f6-87d1-0361d0ae7569 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:11.022079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0c2c7-7711-4104-a878-6b3f1dc1d870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:11.023556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0c2c7-7711-4104-a878-6b3f1dc1d870 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:11.024131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0c2c7-7711-4104-a878-6b3f1dc1d870 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:11.024658 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0c2c7-7711-4104-a878-6b3f1dc1d870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:11.039499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0c2c7-7711-4104-a878-6b3f1dc1d870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:11.046692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0c2c7-7711-4104-a878-6b3f1dc1d870 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:11.052688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1125/2289] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:11.124510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0c2c7-7711-4104-a878-6b3f1dc1d870 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:11.126045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f6c0c2c7-7711-4104-a878-6b3f1dc1d870 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:11.129421 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:11.141249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0c2c7-7711-4104-a878-6b3f1dc1d870 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:11.145529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:11.146567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:11.146920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:11.147268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:11.149771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:11.284378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:11.285902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:11.297858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1126/2290] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:03:11.298723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:11.300095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:11.316352 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144452-0311-43de-94d1-bb89d3407168 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:11.322053 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:11.322957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:11.323352 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:11.323636 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] PATH_INFO: `/v3/projects/6f35a475c4cc4cbcb6e4ca00bf31eab8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:11.324899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f35a475c4cc4cbcb6e4ca00bf31eab8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:11.326886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:11.425071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3174a496203248f38d55e640bcc2cf5b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:11.425071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3174a496203248f38d55e640bcc2cf5b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:11.425071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3174a496203248f38d55e640bcc2cf5b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:11.425071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3174a496203248f38d55e640bcc2cf5b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:11.790935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-81a457ac-f0a4-4523-8912-e3a2b4e1c851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3174a496203248f38d55e640bcc2cf5b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:11.802336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1165/2291] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:10 2022] DELETE /identity/v3/users/3174a496203248f38d55e640bcc2cf5b => generated 0 bytes in 1007 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:03:11.804830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f23a9-7b19-444a-bc07-6c27964d8859 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:11.805296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f23a9-7b19-444a-bc07-6c27964d8859 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:11.805736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f23a9-7b19-444a-bc07-6c27964d8859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:11.816751 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f35a475c4cc4cbcb6e4ca00bf31eab8', 'request_id': 'req-cc164048-8beb-4fe5-b19a-31e9b9806d4d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:11.824130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f35a475c4cc4cbcb6e4ca00bf31eab8', 'request_id': 'req-cc164048-8beb-4fe5-b19a-31e9b9806d4d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:11.827024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-656f23a9-7b19-444a-bc07-6c27964d8859 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:11.828268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-656f23a9-7b19-444a-bc07-6c27964d8859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:11.912626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1166/2292] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:11.924310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f23a9-7b19-444a-bc07-6c27964d8859 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:11.926142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-656f23a9-7b19-444a-bc07-6c27964d8859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:12.002548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f23a9-7b19-444a-bc07-6c27964d8859 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:12.009834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f35a475c4cc4cbcb6e4ca00bf31eab8', 'request_id': 'req-cc164048-8beb-4fe5-b19a-31e9b9806d4d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:12.025135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d340c8-abc7-4e0d-bbca-50070eb2106a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:12.026092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d340c8-abc7-4e0d-bbca-50070eb2106a service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:12.029663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d340c8-abc7-4e0d-bbca-50070eb2106a service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:12.029853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d340c8-abc7-4e0d-bbca-50070eb2106a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:12.030119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] The token cache is being invalidate because project 6f35a475c4cc4cbcb6e4ca00bf31eab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:12.030431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:12.031943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cc164048-8beb-4fe5-b19a-31e9b9806d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:12.035806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1127/2293] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:11 2022] DELETE /identity/v3/projects/6f35a475c4cc4cbcb6e4ca00bf31eab8 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:12.040942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904b7a7-bcfe-4749-8373-da9b41618d22 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:12.041288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904b7a7-bcfe-4749-8373-da9b41618d22 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:12.041569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904b7a7-bcfe-4749-8373-da9b41618d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:12.054519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d340c8-abc7-4e0d-bbca-50070eb2106a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:12.055900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d340c8-abc7-4e0d-bbca-50070eb2106a service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:12.066964 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1167/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:03:12.083849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a904b7a7-bcfe-4749-8373-da9b41618d22 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:12.085357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a904b7a7-bcfe-4749-8373-da9b41618d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:12.106362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6d19-73a1-46fe-8cd8-48dfe1fccb15 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:12.106743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6d19-73a1-46fe-8cd8-48dfe1fccb15 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:12.107346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6d19-73a1-46fe-8cd8-48dfe1fccb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:12.116597 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1128/2295] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:12.129099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d96a6d19-73a1-46fe-8cd8-48dfe1fccb15 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:12.130524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d96a6d19-73a1-46fe-8cd8-48dfe1fccb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:12.148156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904b7a7-bcfe-4749-8373-da9b41618d22 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:12.148284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1168/2296] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:12.172309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904b7a7-bcfe-4749-8373-da9b41618d22 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:12.182271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2010e60-5dbf-455c-90d1-c96448440a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:12.185532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2010e60-5dbf-455c-90d1-c96448440a48 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:12.185971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2010e60-5dbf-455c-90d1-c96448440a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:12.187196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2010e60-5dbf-455c-90d1-c96448440a48 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:12.189956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2010e60-5dbf-455c-90d1-c96448440a48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:12.190434 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2010e60-5dbf-455c-90d1-c96448440a48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:12.197268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2010e60-5dbf-455c-90d1-c96448440a48 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:12.234941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6d19-73a1-46fe-8cd8-48dfe1fccb15 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:12.246648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1129/2297] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:12.247244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6d19-73a1-46fe-8cd8-48dfe1fccb15 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:12.257352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8d1d-a5e1-44e3-b9d6-32384d847bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:12.258450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8d1d-a5e1-44e3-b9d6-32384d847bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:12.258906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8d1d-a5e1-44e3-b9d6-32384d847bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:12.260116 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8d1d-a5e1-44e3-b9d6-32384d847bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:12.265404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea8d1d-a5e1-44e3-b9d6-32384d847bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:12.269179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea8d1d-a5e1-44e3-b9d6-32384d847bc9 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:12.476476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1169/2298] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:12 2022] POST /identity/v3/projects => generated 375 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:12.481399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2010e60-5dbf-455c-90d1-c96448440a48 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:12.531434 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2010e60-5dbf-455c-90d1-c96448440a48 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:12.740550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f982f-13dc-436d-af91-7e52ce71f4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:12.741658 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f982f-13dc-436d-af91-7e52ce71f4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:12.742239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f982f-13dc-436d-af91-7e52ce71f4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:12.744156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f982f-13dc-436d-af91-7e52ce71f4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:12.745047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f982f-13dc-436d-af91-7e52ce71f4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:12.747701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f982f-13dc-436d-af91-7e52ce71f4e2 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:12.848376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:12.885315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1130/2299] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:12 2022] POST /identity/v3/users => generated 288 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:12.889791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8d1d-a5e1-44e3-b9d6-32384d847bc9 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:12.906230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8d1d-a5e1-44e3-b9d6-32384d847bc9 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:12.921168 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf5587-6ead-4065-86b8-cfb72fb037e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:12.923330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf5587-6ead-4065-86b8-cfb72fb037e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:12.923823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf5587-6ead-4065-86b8-cfb72fb037e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:12.924309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf5587-6ead-4065-86b8-cfb72fb037e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:12.931225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f982f-13dc-436d-af91-7e52ce71f4e2 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:12.933474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-875f982f-13dc-436d-af91-7e52ce71f4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:12.936017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf5587-6ead-4065-86b8-cfb72fb037e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:12.937712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf5587-6ead-4065-86b8-cfb72fb037e2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:12.958420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f982f-13dc-436d-af91-7e52ce71f4e2 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:12.964439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:12.967421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:12.967421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:12.967421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] PATH_INFO: `/v3/projects/2bb362ee74e1448b91a735db39b4ac18` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:12.967783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb362ee74e1448b91a735db39b4ac18)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:12.969816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:12.983987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1170/2300] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:12.991730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf5587-6ead-4065-86b8-cfb72fb037e2 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:13.005802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf5587-6ead-4065-86b8-cfb72fb037e2 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:13.012835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777a6af-5ba1-4d4f-87fa-9b32323d192c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:13.013538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777a6af-5ba1-4d4f-87fa-9b32323d192c admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:13.014326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777a6af-5ba1-4d4f-87fa-9b32323d192c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:13.014623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777a6af-5ba1-4d4f-87fa-9b32323d192c admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:13.016366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777a6af-5ba1-4d4f-87fa-9b32323d192c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:13.018588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777a6af-5ba1-4d4f-87fa-9b32323d192c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:13.036390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1171/2301] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:13.046805 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777a6af-5ba1-4d4f-87fa-9b32323d192c admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:13.074105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777a6af-5ba1-4d4f-87fa-9b32323d192c admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:13.075991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716c007-5d55-4405-b16c-4ec32c9f1f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:13.077046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716c007-5d55-4405-b16c-4ec32c9f1f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:13.077400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716c007-5d55-4405-b16c-4ec32c9f1f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:13.077970 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716c007-5d55-4405-b16c-4ec32c9f1f94 admin admin] PATH_INFO: `/v3/projects/80265852e5994f3689366da21c99be5b/users/b43cbcee8821452897e173c6e0c77a82/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:13.108848 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716c007-5d55-4405-b16c-4ec32c9f1f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80265852e5994f3689366da21c99be5b, user_id=b43cbcee8821452897e173c6e0c77a82, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:13.116066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716c007-5d55-4405-b16c-4ec32c9f1f94 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:13.202391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1172/2302] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:13 2022] PUT /identity/v3/projects/80265852e5994f3689366da21c99be5b/users/b43cbcee8821452897e173c6e0c77a82/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:13.205905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d77ffc-af13-4a2d-8120-ecd74535f220 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:13.206468 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d77ffc-af13-4a2d-8120-ecd74535f220 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:13.206965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d77ffc-af13-4a2d-8120-ecd74535f220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:13.222419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb362ee74e1448b91a735db39b4ac18', 'request_id': 'req-4805e05f-582a-4d18-98a5-9fd91bff7f6f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.237301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb362ee74e1448b91a735db39b4ac18', 'request_id': 'req-4805e05f-582a-4d18-98a5-9fd91bff7f6f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.271480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-75d77ffc-af13-4a2d-8120-ecd74535f220 None None] MFA Rules not processed for user `b43cbcee8821452897e173c6e0c77a82`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:13.273306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-75d77ffc-af13-4a2d-8120-ecd74535f220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:13.333678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb362ee74e1448b91a735db39b4ac18', 'request_id': 'req-4805e05f-582a-4d18-98a5-9fd91bff7f6f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.353331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] The token cache is being invalidate because project 2bb362ee74e1448b91a735db39b4ac18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:13.353632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.354878 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.358659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1131/2303] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:12 2022] DELETE /identity/v3/projects/2bb362ee74e1448b91a735db39b4ac18 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:13.364014 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:13.373033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:13.402365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1173/2304] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:03:13.417787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d77ffc-af13-4a2d-8120-ecd74535f220 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:13.420069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-75d77ffc-af13-4a2d-8120-ecd74535f220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:13.499377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805e05f-582a-4d18-98a5-9fd91bff7f6f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:13.505257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:13.506268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:13.506631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:13.506986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] PATH_INFO: `/v3/users/cfc4e417c4c146adacb1927dc0e875e6` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:13.508537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc4e417c4c146adacb1927dc0e875e6)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:13.511534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:13.545801 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d77ffc-af13-4a2d-8120-ecd74535f220 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:13.559629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644575e0-6eeb-4fdb-9994-d415ec2f36b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:13.560675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644575e0-6eeb-4fdb-9994-d415ec2f36b6 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:13.561170 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644575e0-6eeb-4fdb-9994-d415ec2f36b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:13.561629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644575e0-6eeb-4fdb-9994-d415ec2f36b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:13.576443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644575e0-6eeb-4fdb-9994-d415ec2f36b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:13.578244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644575e0-6eeb-4fdb-9994-d415ec2f36b6 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:13.592441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1174/2305] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:13.761914 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc4e417c4c146adacb1927dc0e875e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.771752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc4e417c4c146adacb1927dc0e875e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.772891 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc4e417c4c146adacb1927dc0e875e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.854242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc4e417c4c146adacb1927dc0e875e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.873776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc4e417c4c146adacb1927dc0e875e6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:13.880106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1132/2306] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:13 2022] DELETE /identity/v3/users/cfc4e417c4c146adacb1927dc0e875e6 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:13.948925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:14.358936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644575e0-6eeb-4fdb-9994-d415ec2f36b6 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:14.378951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644575e0-6eeb-4fdb-9994-d415ec2f36b6 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:14.536255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:14.537461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:14.537580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:14.537827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] PATH_INFO: `/v3/projects/ef61d7d231934295b3584ccdddf50b26` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:14.539054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef61d7d231934295b3584ccdddf50b26)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:14.541132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:14.557495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:14.697089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef61d7d231934295b3584ccdddf50b26', 'request_id': 'req-6b098329-e2cc-4083-beda-dbc042ecf2b1'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:14.702828 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef61d7d231934295b3584ccdddf50b26', 'request_id': 'req-6b098329-e2cc-4083-beda-dbc042ecf2b1'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:14.714927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef61d7d231934295b3584ccdddf50b26', 'request_id': 'req-6b098329-e2cc-4083-beda-dbc042ecf2b1'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:14.792787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:14.800446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] The token cache is being invalidate because project ef61d7d231934295b3584ccdddf50b26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:14.803294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:14.803294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:14.815059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1175/2307] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:14 2022] DELETE /identity/v3/projects/ef61d7d231934295b3584ccdddf50b26 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:14.818885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:14.820656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:14.837735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432f36c-0e61-4a47-b66d-c18fdb8c589e admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:14.905655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:14.906557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:14.907030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:14.907310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] PATH_INFO: `/v3/users/083f1b8c7d5247ac8655bcf8b5564bf3` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:14.908542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=083f1b8c7d5247ac8655bcf8b5564bf3)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:14.910545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:15.086648 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083f1b8c7d5247ac8655bcf8b5564bf3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:15.096993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083f1b8c7d5247ac8655bcf8b5564bf3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:15.097754 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083f1b8c7d5247ac8655bcf8b5564bf3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:15.156487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083f1b8c7d5247ac8655bcf8b5564bf3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:15.171987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0f33e720-91e0-4284-872c-68616328df27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083f1b8c7d5247ac8655bcf8b5564bf3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:15.178881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1133/2308] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:14 2022] DELETE /identity/v3/users/083f1b8c7d5247ac8655bcf8b5564bf3 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:16.350045 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:16.355044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:16.396885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b098329-e2cc-4083-beda-dbc042ecf2b1 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:16.500153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:16.501142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:16.501456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:16.501743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] PATH_INFO: `/v3/projects/60dd86f5ec50424c9426cbb039ffc1e5` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:16.502945 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=60dd86f5ec50424c9426cbb039ffc1e5)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:16.505460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:16.580011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:16.617290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60dd86f5ec50424c9426cbb039ffc1e5', 'request_id': 'req-691ca077-65f5-4a2d-a48f-90ecedee8f2a'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:16.625856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60dd86f5ec50424c9426cbb039ffc1e5', 'request_id': 'req-691ca077-65f5-4a2d-a48f-90ecedee8f2a'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:16.637363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60dd86f5ec50424c9426cbb039ffc1e5', 'request_id': 'req-691ca077-65f5-4a2d-a48f-90ecedee8f2a'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:16.677072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:16.741892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] The token cache is being invalidate because project 60dd86f5ec50424c9426cbb039ffc1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:16.742192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:16.742892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-691ca077-65f5-4a2d-a48f-90ecedee8f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:16.745362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1176/2309] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:16 2022] DELETE /identity/v3/projects/60dd86f5ec50424c9426cbb039ffc1e5 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:16.750317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a56986-cedc-4108-b4ab-fa82a65a87e6 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:16.750614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a56986-cedc-4108-b4ab-fa82a65a87e6 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:16.750902 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a56986-cedc-4108-b4ab-fa82a65a87e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:16.785064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-70a56986-cedc-4108-b4ab-fa82a65a87e6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:16.788707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-70a56986-cedc-4108-b4ab-fa82a65a87e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:16.881174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1134/2310] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:03:16.882312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33ee8d-fd4f-4581-a4a0-9bce59adc7e6 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:16.882633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33ee8d-fd4f-4581-a4a0-9bce59adc7e6 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:16.882969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33ee8d-fd4f-4581-a4a0-9bce59adc7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:16.925167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-8f33ee8d-fd4f-4581-a4a0-9bce59adc7e6 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:16.933092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8f33ee8d-fd4f-4581-a4a0-9bce59adc7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:16.956298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1177/2311] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:16.957293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a56986-cedc-4108-b4ab-fa82a65a87e6 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:16.975537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a56986-cedc-4108-b4ab-fa82a65a87e6 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:16.986633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0fa68-da95-43fa-92df-b204a78ae510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:16.987935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0fa68-da95-43fa-92df-b204a78ae510 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:16.988324 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0fa68-da95-43fa-92df-b204a78ae510 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:16.988742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0fa68-da95-43fa-92df-b204a78ae510 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:16.992598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0fa68-da95-43fa-92df-b204a78ae510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:16.992598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0fa68-da95-43fa-92df-b204a78ae510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:16.992598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0fa68-da95-43fa-92df-b204a78ae510 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:17.019838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33ee8d-fd4f-4581-a4a0-9bce59adc7e6 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:17.026442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1135/2312] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:17.030487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33ee8d-fd4f-4581-a4a0-9bce59adc7e6 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:17.038886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c167848-249a-49ae-a2a1-1343ad6623aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:17.040393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c167848-249a-49ae-a2a1-1343ad6623aa admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:17.040947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c167848-249a-49ae-a2a1-1343ad6623aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:17.041433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c167848-249a-49ae-a2a1-1343ad6623aa admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:17.042514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c167848-249a-49ae-a2a1-1343ad6623aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:17.044717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c167848-249a-49ae-a2a1-1343ad6623aa admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:17.153977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1178/2313] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:17 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:17.154064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0fa68-da95-43fa-92df-b204a78ae510 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:17.159571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0fa68-da95-43fa-92df-b204a78ae510 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:17.319144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21974d4-ae8a-4f7a-8df5-262170fef9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:17.320243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21974d4-ae8a-4f7a-8df5-262170fef9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:17.320630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21974d4-ae8a-4f7a-8df5-262170fef9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:17.321041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21974d4-ae8a-4f7a-8df5-262170fef9a1 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:17.321934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21974d4-ae8a-4f7a-8df5-262170fef9a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:17.324548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21974d4-ae8a-4f7a-8df5-262170fef9a1 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:17.443293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c167848-249a-49ae-a2a1-1343ad6623aa admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:17.449036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1136/2314] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:17 2022] POST /identity/v3/users => generated 286 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:17.456913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c167848-249a-49ae-a2a1-1343ad6623aa admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:17.462841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed6b48-7836-4f54-97c5-bb918d6055fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:17.463616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed6b48-7836-4f54-97c5-bb918d6055fa admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:17.464186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed6b48-7836-4f54-97c5-bb918d6055fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:17.464270 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed6b48-7836-4f54-97c5-bb918d6055fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:17.465150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed6b48-7836-4f54-97c5-bb918d6055fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:17.466946 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed6b48-7836-4f54-97c5-bb918d6055fa admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:17.482972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1179/2315] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:17.483784 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21974d4-ae8a-4f7a-8df5-262170fef9a1 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:17.498214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21974d4-ae8a-4f7a-8df5-262170fef9a1 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:17.509407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf975f60-aa8a-4d68-9975-1806fcd1bae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:17.510165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf975f60-aa8a-4d68-9975-1806fcd1bae5 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:17.510462 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf975f60-aa8a-4d68-9975-1806fcd1bae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:17.510747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf975f60-aa8a-4d68-9975-1806fcd1bae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:17.511507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf975f60-aa8a-4d68-9975-1806fcd1bae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:17.513334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf975f60-aa8a-4d68-9975-1806fcd1bae5 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:17.528837 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1137/2316] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:03:17.530764 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed6b48-7836-4f54-97c5-bb918d6055fa admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:17.545142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed6b48-7836-4f54-97c5-bb918d6055fa admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:17.558796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b644d8-b6db-4f5c-b49a-5543c2589d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:17.559699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b644d8-b6db-4f5c-b49a-5543c2589d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:17.559986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b644d8-b6db-4f5c-b49a-5543c2589d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:17.560262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b644d8-b6db-4f5c-b49a-5543c2589d20 admin admin] PATH_INFO: `/v3/projects/d130b3a0b56e4381a39350b966f5c29a/users/77d902e9f51f433880a6ec574a9abfa1/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:17.593380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b644d8-b6db-4f5c-b49a-5543c2589d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d130b3a0b56e4381a39350b966f5c29a, user_id=77d902e9f51f433880a6ec574a9abfa1, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:17.596396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b644d8-b6db-4f5c-b49a-5543c2589d20 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:17.684303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1180/2317] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:17 2022] PUT /identity/v3/projects/d130b3a0b56e4381a39350b966f5c29a/users/77d902e9f51f433880a6ec574a9abfa1/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:17.696070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ab56d-a999-4a9f-b555-338ebebf048e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:17.697226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ab56d-a999-4a9f-b555-338ebebf048e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:17.697226 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ab56d-a999-4a9f-b555-338ebebf048e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:17.754239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-345ab56d-a999-4a9f-b555-338ebebf048e None None] MFA Rules not processed for user `77d902e9f51f433880a6ec574a9abfa1`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:17.755641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-345ab56d-a999-4a9f-b555-338ebebf048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:17.851851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1138/2318] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:17.866229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b644d8-b6db-4f5c-b49a-5543c2589d20 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:17.869744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-98b644d8-b6db-4f5c-b49a-5543c2589d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:17.962795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b644d8-b6db-4f5c-b49a-5543c2589d20 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:17.974573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a957ebe-be87-47ed-914c-08cda0d747b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca33ef49ba44832a7fb45b2317d211c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:17.975680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957ebe-be87-47ed-914c-08cda0d747b9 service glance] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:17.976175 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957ebe-be87-47ed-914c-08cda0d747b9 service glance] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:17.976670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957ebe-be87-47ed-914c-08cda0d747b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:17.992174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957ebe-be87-47ed-914c-08cda0d747b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:17.996023 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957ebe-be87-47ed-914c-08cda0d747b9 service glance] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:17.997808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:18.022191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1181/2319] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:18.786563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:19.300386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:21.329492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:21.403618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1f0bf-4dc1-4b03-9559-7dc87ef11366 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:21.403618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1f0bf-4dc1-4b03-9559-7dc87ef11366 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:21.403618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1f0bf-4dc1-4b03-9559-7dc87ef11366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:21.434705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f3a1f0bf-4dc1-4b03-9559-7dc87ef11366 None None] MFA Rules not processed for user `5221431e0552435d9ff82f8ce75b624e`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:21.436015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f3a1f0bf-4dc1-4b03-9559-7dc87ef11366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:21.515685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1139/2320] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:21.785950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34865ab8-fa4a-4e76-bc67-e8381fdb69b0 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:21.786138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34865ab8-fa4a-4e76-bc67-e8381fdb69b0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:21.786680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34865ab8-fa4a-4e76-bc67-e8381fdb69b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:21.813867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-34865ab8-fa4a-4e76-bc67-e8381fdb69b0 None None] MFA Rules not processed for user `3f1b6145d67d4cbf9dcd398736e6b0da`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:21.815539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-34865ab8-fa4a-4e76-bc67-e8381fdb69b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:21.888600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1182/2321] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:22.090589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:22.867456 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:23.608795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:23.677666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:24.758963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1f0bf-4dc1-4b03-9559-7dc87ef11366 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:24.760982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f3a1f0bf-4dc1-4b03-9559-7dc87ef11366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:24.825414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1f0bf-4dc1-4b03-9559-7dc87ef11366 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:24.829965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:24.830835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:24.831124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:24.831404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] PATH_INFO: `/v3/users/7288c84d23434997acbf3f0af85a9f99` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:24.832598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7288c84d23434997acbf3f0af85a9f99)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:24.834697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:24.957236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:25.041405 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7288c84d23434997acbf3f0af85a9f99'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:25.048922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7288c84d23434997acbf3f0af85a9f99'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:25.049697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7288c84d23434997acbf3f0af85a9f99'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:25.082730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:25.099418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7288c84d23434997acbf3f0af85a9f99'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:25.113821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7288c84d23434997acbf3f0af85a9f99'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:25.122070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1140/2322] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:24 2022] DELETE /identity/v3/users/7288c84d23434997acbf3f0af85a9f99 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:25.127009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb748795-a684-4410-8869-07f20e6fc154 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:25.127522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb748795-a684-4410-8869-07f20e6fc154 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:25.127873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb748795-a684-4410-8869-07f20e6fc154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:25.162994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-fb748795-a684-4410-8869-07f20e6fc154 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:25.164319 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fb748795-a684-4410-8869-07f20e6fc154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:25.243279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1183/2323] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:25.253502 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:25.254986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:25.288761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:25.314394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0a9c0-fa3e-4e3e-ada5-5f976d726d65 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:25.318432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1663a09-2970-40d1-b316-3cbbf0b19d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:25.319121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1663a09-2970-40d1-b316-3cbbf0b19d1b service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:25.319403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1663a09-2970-40d1-b316-3cbbf0b19d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:25.319677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1663a09-2970-40d1-b316-3cbbf0b19d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:25.329173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1663a09-2970-40d1-b316-3cbbf0b19d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:25.330681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1663a09-2970-40d1-b316-3cbbf0b19d1b service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:25.340089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1141/2324] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:25.768308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:25.799201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb748795-a684-4410-8869-07f20e6fc154 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:25.800734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fb748795-a684-4410-8869-07f20e6fc154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:25.810124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:25.815648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb748795-a684-4410-8869-07f20e6fc154 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:25.819842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:25.820747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:25.821042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:25.821322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] PATH_INFO: `/v3/projects/c6106147a3224d148e36e1c1a8b8d6e5` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:25.822512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6106147a3224d148e36e1c1a8b8d6e5)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:25.824412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:25.903757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6106147a3224d148e36e1c1a8b8d6e5', 'request_id': 'req-8a29c552-3c3f-4543-99ab-474b96dd2408'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:25.909560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6106147a3224d148e36e1c1a8b8d6e5', 'request_id': 'req-8a29c552-3c3f-4543-99ab-474b96dd2408'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:25.918326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6106147a3224d148e36e1c1a8b8d6e5', 'request_id': 'req-8a29c552-3c3f-4543-99ab-474b96dd2408'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.024602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] The token cache is being invalidate because project c6106147a3224d148e36e1c1a8b8d6e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:26.025578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.026195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.027786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1184/2325] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:25 2022] DELETE /identity/v3/projects/c6106147a3224d148e36e1c1a8b8d6e5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:26.031825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1663a09-2970-40d1-b316-3cbbf0b19d1b service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:26.042076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1663a09-2970-40d1-b316-3cbbf0b19d1b service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:26.044668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.077015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.106239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:26.107127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:26.107413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:26.107690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] PATH_INFO: `/v3/users/14bfe5bc92814dcc983da69ae6bcfb7e` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:26.108958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=14bfe5bc92814dcc983da69ae6bcfb7e)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:26.110975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:26.257319 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14bfe5bc92814dcc983da69ae6bcfb7e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.266365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.272377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14bfe5bc92814dcc983da69ae6bcfb7e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.273735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14bfe5bc92814dcc983da69ae6bcfb7e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.305670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.328130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14bfe5bc92814dcc983da69ae6bcfb7e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.343606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-340a7e9a-a639-49a9-9e47-438606ff4a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14bfe5bc92814dcc983da69ae6bcfb7e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.349346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1142/2326] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:26 2022] DELETE /identity/v3/users/14bfe5bc92814dcc983da69ae6bcfb7e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:26.432916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.477060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.545369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.565773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.747609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:26.749110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:26.762925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c552-3c3f-4543-99ab-474b96dd2408 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:26.780029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.821395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:26.822323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:26.822625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:26.822930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] PATH_INFO: `/v3/projects/1eec27259eb148c6b95a9844af6ede1a` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:26.824893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eec27259eb148c6b95a9844af6ede1a)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:26.827441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:26.830688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:26.946183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eec27259eb148c6b95a9844af6ede1a', 'request_id': 'req-99d7c1e0-2989-44df-98e3-f553cafa1020'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.951576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eec27259eb148c6b95a9844af6ede1a', 'request_id': 'req-99d7c1e0-2989-44df-98e3-f553cafa1020'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:26.957678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eec27259eb148c6b95a9844af6ede1a', 'request_id': 'req-99d7c1e0-2989-44df-98e3-f553cafa1020'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:27.024527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] The token cache is being invalidate because project 1eec27259eb148c6b95a9844af6ede1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:27.024880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:27.025501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-99d7c1e0-2989-44df-98e3-f553cafa1020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:27.027057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1185/2327] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:26 2022] DELETE /identity/v3/projects/1eec27259eb148c6b95a9844af6ede1a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:27.031962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232978c3-cf40-494c-acd3-700d94aaf3fb None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.032265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232978c3-cf40-494c-acd3-700d94aaf3fb None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.032547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232978c3-cf40-494c-acd3-700d94aaf3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.051400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-232978c3-cf40-494c-acd3-700d94aaf3fb None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:27.052613 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-232978c3-cf40-494c-acd3-700d94aaf3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:27.080160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:27.114236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1143/2328] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:27.122664 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8e8d7-6c5a-4bab-90fb-9de44411a1ac None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.122963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8e8d7-6c5a-4bab-90fb-9de44411a1ac None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.123335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8e8d7-6c5a-4bab-90fb-9de44411a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.142222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e4d8e8d7-6c5a-4bab-90fb-9de44411a1ac None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:27.143366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e4d8e8d7-6c5a-4bab-90fb-9de44411a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:27.152035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1186/2329] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:27.156145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232978c3-cf40-494c-acd3-700d94aaf3fb None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:27.167551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232978c3-cf40-494c-acd3-700d94aaf3fb None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:27.172536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbaa3f-92a2-41b8-9a01-449dacf6943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:27.173244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbaa3f-92a2-41b8-9a01-449dacf6943a admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.173789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbaa3f-92a2-41b8-9a01-449dacf6943a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.174072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbaa3f-92a2-41b8-9a01-449dacf6943a admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.174810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbaa3f-92a2-41b8-9a01-449dacf6943a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:27.175168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbaa3f-92a2-41b8-9a01-449dacf6943a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:27.176832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbaa3f-92a2-41b8-9a01-449dacf6943a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:27.190034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1144/2330] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:27.194363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8e8d7-6c5a-4bab-90fb-9de44411a1ac None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:27.202478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:27.205968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8e8d7-6c5a-4bab-90fb-9de44411a1ac None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:27.210822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435929b-752d-453c-872a-ae59450b2747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:27.211484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435929b-752d-453c-872a-ae59450b2747 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.211768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435929b-752d-453c-872a-ae59450b2747 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.213019 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435929b-752d-453c-872a-ae59450b2747 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.215449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435929b-752d-453c-872a-ae59450b2747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:27.215449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435929b-752d-453c-872a-ae59450b2747 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:27.309128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1187/2331] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:27 2022] POST /identity/v3/projects => generated 407 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:27.313422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbaa3f-92a2-41b8-9a01-449dacf6943a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:27.325144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbaa3f-92a2-41b8-9a01-449dacf6943a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:27.356633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:27.388487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56483cda-d39f-45df-88de-a8ec4897824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:27.389207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483cda-d39f-45df-88de-a8ec4897824f admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.389478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483cda-d39f-45df-88de-a8ec4897824f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.389756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483cda-d39f-45df-88de-a8ec4897824f admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.390447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56483cda-d39f-45df-88de-a8ec4897824f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:27.392437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56483cda-d39f-45df-88de-a8ec4897824f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:27.511885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1145/2332] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:27 2022] POST /identity/v3/users => generated 304 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:03:27.515972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435929b-752d-453c-872a-ae59450b2747 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:27.522278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:27.527223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435929b-752d-453c-872a-ae59450b2747 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:27.532133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28b29e-3c2e-46ec-96ab-f7f8aad48539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:27.532879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28b29e-3c2e-46ec-96ab-f7f8aad48539 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.533174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28b29e-3c2e-46ec-96ab-f7f8aad48539 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.533512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28b29e-3c2e-46ec-96ab-f7f8aad48539 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.534269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28b29e-3c2e-46ec-96ab-f7f8aad48539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:27.535879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28b29e-3c2e-46ec-96ab-f7f8aad48539 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:27.547576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1188/2333] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:27.551950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56483cda-d39f-45df-88de-a8ec4897824f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:27.562373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56483cda-d39f-45df-88de-a8ec4897824f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:27.567239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dedda-4b23-49b7-88cd-ff10b6e14df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:27.567917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dedda-4b23-49b7-88cd-ff10b6e14df1 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.568209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dedda-4b23-49b7-88cd-ff10b6e14df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.568500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dedda-4b23-49b7-88cd-ff10b6e14df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.569499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dedda-4b23-49b7-88cd-ff10b6e14df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:27.572307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dedda-4b23-49b7-88cd-ff10b6e14df1 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:27.586276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1146/2334] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:27.592453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28b29e-3c2e-46ec-96ab-f7f8aad48539 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:27.612097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28b29e-3c2e-46ec-96ab-f7f8aad48539 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:27.613726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7435135-9d63-4534-9d3b-381b3be89daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:27.615175 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7435135-9d63-4534-9d3b-381b3be89daf admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.615667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7435135-9d63-4534-9d3b-381b3be89daf admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.615975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7435135-9d63-4534-9d3b-381b3be89daf admin admin] PATH_INFO: `/v3/projects/d26f18f9c1b442c1b6d03dd3f4982bd3/users/18255f3f84da457a9bd3db414a49a84f/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.636934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7435135-9d63-4534-9d3b-381b3be89daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d26f18f9c1b442c1b6d03dd3f4982bd3, user_id=18255f3f84da457a9bd3db414a49a84f, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:27.639626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7435135-9d63-4534-9d3b-381b3be89daf admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:27.705203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1189/2335] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:27 2022] PUT /identity/v3/projects/d26f18f9c1b442c1b6d03dd3f4982bd3/users/18255f3f84da457a9bd3db414a49a84f/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:27.717168 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dee85-ed38-4b10-af11-567b2219eca8 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.717466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dee85-ed38-4b10-af11-567b2219eca8 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.717746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dee85-ed38-4b10-af11-567b2219eca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.747872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b67dee85-ed38-4b10-af11-567b2219eca8 None None] MFA Rules not processed for user `18255f3f84da457a9bd3db414a49a84f`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:27.749043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b67dee85-ed38-4b10-af11-567b2219eca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:27.813857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1147/2336] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:27 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:27.818355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7435135-9d63-4534-9d3b-381b3be89daf admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:27.836101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:27.882779 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7435135-9d63-4534-9d3b-381b3be89daf admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:27.890594 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5e759-16c2-4645-a0b6-0e6d643bb888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:27.891280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5e759-16c2-4645-a0b6-0e6d643bb888 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:27.892824 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5e759-16c2-4645-a0b6-0e6d643bb888 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:27.893127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5e759-16c2-4645-a0b6-0e6d643bb888 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:27.894261 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5e759-16c2-4645-a0b6-0e6d643bb888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:27.898369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5e759-16c2-4645-a0b6-0e6d643bb888 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:28.019422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1190/2337] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:27 2022] POST /identity/v3/projects => generated 409 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:28.020554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dee85-ed38-4b10-af11-567b2219eca8 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:28.031235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dee85-ed38-4b10-af11-567b2219eca8 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:28.059747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:28.087208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ead9e3-1729-4949-a859-1bb3c4f34bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:28.087886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ead9e3-1729-4949-a859-1bb3c4f34bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:28.088170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ead9e3-1729-4949-a859-1bb3c4f34bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:28.088449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ead9e3-1729-4949-a859-1bb3c4f34bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:28.089175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ead9e3-1729-4949-a859-1bb3c4f34bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:28.091074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ead9e3-1729-4949-a859-1bb3c4f34bd1 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:28.157397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1148/2338] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:28 2022] POST /identity/v3/users => generated 311 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:28.161559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5e759-16c2-4645-a0b6-0e6d643bb888 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:28.172324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5e759-16c2-4645-a0b6-0e6d643bb888 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:28.177375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d8ac-3d65-421b-adbd-f4b2fadb9bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:28.178036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d8ac-3d65-421b-adbd-f4b2fadb9bef admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:28.178324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d8ac-3d65-421b-adbd-f4b2fadb9bef admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:28.178604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d8ac-3d65-421b-adbd-f4b2fadb9bef admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:28.179375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53d8ac-3d65-421b-adbd-f4b2fadb9bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:28.180993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53d8ac-3d65-421b-adbd-f4b2fadb9bef admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:28.193864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1191/2339] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:28.198724 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ead9e3-1729-4949-a859-1bb3c4f34bd1 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:28.208970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ead9e3-1729-4949-a859-1bb3c4f34bd1 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:28.213646 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd3af5-6b4c-4b72-81a9-f96c291c71a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:28.214545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd3af5-6b4c-4b72-81a9-f96c291c71a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:28.214829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd3af5-6b4c-4b72-81a9-f96c291c71a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:28.215106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd3af5-6b4c-4b72-81a9-f96c291c71a9 admin admin] PATH_INFO: `/v3/projects/5d2dd768007e4dab983ec519022dd7e0/users/4cf17b0dd2fc4e0e8749dc939d73a620/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:28.231468 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd3af5-6b4c-4b72-81a9-f96c291c71a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2dd768007e4dab983ec519022dd7e0, user_id=4cf17b0dd2fc4e0e8749dc939d73a620, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:28.234315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd3af5-6b4c-4b72-81a9-f96c291c71a9 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:28.300314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1149/2340] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:28 2022] PUT /identity/v3/projects/5d2dd768007e4dab983ec519022dd7e0/users/4cf17b0dd2fc4e0e8749dc939d73a620/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:28.304542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d8ac-3d65-421b-adbd-f4b2fadb9bef admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:28.315288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d8ac-3d65-421b-adbd-f4b2fadb9bef admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:28.375496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8fdf0-128d-4dd7-a07e-ceb5f62788ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:28.376830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8fdf0-128d-4dd7-a07e-ceb5f62788ea admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:28.376830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8fdf0-128d-4dd7-a07e-ceb5f62788ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:28.377103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8fdf0-128d-4dd7-a07e-ceb5f62788ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:28.377698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8fdf0-128d-4dd7-a07e-ceb5f62788ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:28.379302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8fdf0-128d-4dd7-a07e-ceb5f62788ea admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:28.392996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1192/2341] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:03:28.396694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd3af5-6b4c-4b72-81a9-f96c291c71a9 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:28.414166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd3af5-6b4c-4b72-81a9-f96c291c71a9 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:28.419380 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95228a6-17dc-4415-90e0-107ce38a55bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:28.420301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95228a6-17dc-4415-90e0-107ce38a55bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:28.420588 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95228a6-17dc-4415-90e0-107ce38a55bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:28.420911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95228a6-17dc-4415-90e0-107ce38a55bb admin admin] PATH_INFO: `/v3/projects/5d2dd768007e4dab983ec519022dd7e0/users/4cf17b0dd2fc4e0e8749dc939d73a620/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:28.423010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95228a6-17dc-4415-90e0-107ce38a55bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2dd768007e4dab983ec519022dd7e0, user_id=4cf17b0dd2fc4e0e8749dc939d73a620, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:28.425962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95228a6-17dc-4415-90e0-107ce38a55bb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:28.438332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1150/2342] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:28 2022] PUT /identity/v3/projects/5d2dd768007e4dab983ec519022dd7e0/users/4cf17b0dd2fc4e0e8749dc939d73a620/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:28.448567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a37347-2826-4767-b801-7056c18f9a04 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:28.448959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a37347-2826-4767-b801-7056c18f9a04 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:28.449309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a37347-2826-4767-b801-7056c18f9a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:28.482025 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d6a37347-2826-4767-b801-7056c18f9a04 None None] MFA Rules not processed for user `4cf17b0dd2fc4e0e8749dc939d73a620`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:28.483406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d6a37347-2826-4767-b801-7056c18f9a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:28.552195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1193/2343] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:28 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:28.565608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95228a6-17dc-4415-90e0-107ce38a55bb admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:28.567591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e95228a6-17dc-4415-90e0-107ce38a55bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:28.625007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95228a6-17dc-4415-90e0-107ce38a55bb admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:28.628630 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb193d0-dace-49e3-be79-c8a049c09965 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:28.629295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb193d0-dace-49e3-be79-c8a049c09965 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:28.629577 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb193d0-dace-49e3-be79-c8a049c09965 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:28.629853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb193d0-dace-49e3-be79-c8a049c09965 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:28.639892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb193d0-dace-49e3-be79-c8a049c09965 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:28.641344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb193d0-dace-49e3-be79-c8a049c09965 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:28.652815 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1151/2344] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:28 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:29.093375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a37347-2826-4767-b801-7056c18f9a04 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:29.104707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a37347-2826-4767-b801-7056c18f9a04 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:29.109296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828951a-2962-44e3-bed0-5eb3b4629413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:29.109991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828951a-2962-44e3-bed0-5eb3b4629413 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:29.110326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828951a-2962-44e3-bed0-5eb3b4629413 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:29.110607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828951a-2962-44e3-bed0-5eb3b4629413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:29.122583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:29.185028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828951a-2962-44e3-bed0-5eb3b4629413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:29.186626 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828951a-2962-44e3-bed0-5eb3b4629413 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:29.198182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1194/2345] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:29 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:30.313114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a8fd-696a-444f-a0a2-295ac32630e2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:30.314000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a8fd-696a-444f-a0a2-295ac32630e2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:30.314362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a8fd-696a-444f-a0a2-295ac32630e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:30.335072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3c87a8fd-696a-444f-a0a2-295ac32630e2 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:30.336457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3c87a8fd-696a-444f-a0a2-295ac32630e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:30.406768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1152/2346] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:03:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:30.422196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828951a-2962-44e3-bed0-5eb3b4629413 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:30.424009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2828951a-2962-44e3-bed0-5eb3b4629413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:30.517788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828951a-2962-44e3-bed0-5eb3b4629413 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:30.525387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711977df-bfbe-4d14-b890-acfb672eca1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:30.526148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711977df-bfbe-4d14-b890-acfb672eca1d service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:30.526496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711977df-bfbe-4d14-b890-acfb672eca1d service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:30.527354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711977df-bfbe-4d14-b890-acfb672eca1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:30.536342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711977df-bfbe-4d14-b890-acfb672eca1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:30.538223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711977df-bfbe-4d14-b890-acfb672eca1d service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:30.547651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1195/2347] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:03:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:03:37.046480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a8fd-696a-444f-a0a2-295ac32630e2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:37.049891 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3c87a8fd-696a-444f-a0a2-295ac32630e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:37.115298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:37.187850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a8fd-696a-444f-a0a2-295ac32630e2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:37.195201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:37.196494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:37.197046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:37.197496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] PATH_INFO: `/v3/users/fde9403ad838416abc7cdab4d9405362` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:37.199329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde9403ad838416abc7cdab4d9405362)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:37.201836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:37.458403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde9403ad838416abc7cdab4d9405362'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:37.501848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde9403ad838416abc7cdab4d9405362'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:37.506387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde9403ad838416abc7cdab4d9405362'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:37.980718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde9403ad838416abc7cdab4d9405362'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:37.996061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde9403ad838416abc7cdab4d9405362'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:38.002718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1153/2348] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:37 2022] DELETE /identity/v3/users/fde9403ad838416abc7cdab4d9405362 => generated 0 bytes in 957 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:38.007343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b178268-3e0f-4381-9756-566a9f7aacbe None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:38.008225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b178268-3e0f-4381-9756-566a9f7aacbe None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:38.010537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b178268-3e0f-4381-9756-566a9f7aacbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:38.031447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-7b178268-3e0f-4381-9756-566a9f7aacbe None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:38.032703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7b178268-3e0f-4381-9756-566a9f7aacbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:38.153404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1196/2349] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:03:38.173143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:38.175401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:38.310249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d875-f0c5-4a9a-b9e9-3c8b305b0665 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:38.320425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc26f8-84b8-4a29-a104-1dc0a6ebce01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:38.321144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc26f8-84b8-4a29-a104-1dc0a6ebce01 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:38.321425 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc26f8-84b8-4a29-a104-1dc0a6ebce01 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:38.321832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc26f8-84b8-4a29-a104-1dc0a6ebce01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:38.362601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc26f8-84b8-4a29-a104-1dc0a6ebce01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:38.370936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc26f8-84b8-4a29-a104-1dc0a6ebce01 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:38.405578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1154/2350] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:39.096628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b178268-3e0f-4381-9756-566a9f7aacbe None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:39.102666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7b178268-3e0f-4381-9756-566a9f7aacbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:39.121403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b178268-3e0f-4381-9756-566a9f7aacbe None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:39.135503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:39.136584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:39.137321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:39.137817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] PATH_INFO: `/v3/projects/3d603cf539d940d39df533ae9dece453` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:39.146528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d603cf539d940d39df533ae9dece453)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:39.153878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:39.300066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:39.312787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d603cf539d940d39df533ae9dece453', 'request_id': 'req-fcdf617b-9f9e-43e3-ac95-efb862614f96'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:39.322561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d603cf539d940d39df533ae9dece453', 'request_id': 'req-fcdf617b-9f9e-43e3-ac95-efb862614f96'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:39.332688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d603cf539d940d39df533ae9dece453', 'request_id': 'req-fcdf617b-9f9e-43e3-ac95-efb862614f96'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:39.420303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] The token cache is being invalidate because project 3d603cf539d940d39df533ae9dece453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:39.422093 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:39.422093 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fcdf617b-9f9e-43e3-ac95-efb862614f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:39.429391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1197/2351] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:39 2022] DELETE /identity/v3/projects/3d603cf539d940d39df533ae9dece453 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:39.431853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7690-b64e-4cc7-a940-3124ba0a59b0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:39.432768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7690-b64e-4cc7-a940-3124ba0a59b0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:39.433222 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7690-b64e-4cc7-a940-3124ba0a59b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:39.466365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-855d7690-b64e-4cc7-a940-3124ba0a59b0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:39.468070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-855d7690-b64e-4cc7-a940-3124ba0a59b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:39.604011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:39.624688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1155/2352] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:39.630050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663788ac-11a4-444e-8cb5-0472204b68a2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:39.633891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663788ac-11a4-444e-8cb5-0472204b68a2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:39.633891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663788ac-11a4-444e-8cb5-0472204b68a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:39.656021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-663788ac-11a4-444e-8cb5-0472204b68a2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:39.657489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-663788ac-11a4-444e-8cb5-0472204b68a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:39.666968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1198/2353] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:39.671510 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d7690-b64e-4cc7-a940-3124ba0a59b0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:39.684699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d7690-b64e-4cc7-a940-3124ba0a59b0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:39.691651 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c37dd-4eee-41e2-ac23-9bf6e30c4efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:39.693202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c37dd-4eee-41e2-ac23-9bf6e30c4efb admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:39.693640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c37dd-4eee-41e2-ac23-9bf6e30c4efb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:39.693928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c37dd-4eee-41e2-ac23-9bf6e30c4efb admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:39.694904 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c37dd-4eee-41e2-ac23-9bf6e30c4efb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:39.695245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c37dd-4eee-41e2-ac23-9bf6e30c4efb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:39.697478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c37dd-4eee-41e2-ac23-9bf6e30c4efb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:39.711240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1156/2354] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:39.717375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663788ac-11a4-444e-8cb5-0472204b68a2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:39.731505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663788ac-11a4-444e-8cb5-0472204b68a2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:39.739742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602a631-1fcb-4693-b59d-6964b5c0d0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:39.740550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602a631-1fcb-4693-b59d-6964b5c0d0ee admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:39.740899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602a631-1fcb-4693-b59d-6964b5c0d0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:39.741231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602a631-1fcb-4693-b59d-6964b5c0d0ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:39.741963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602a631-1fcb-4693-b59d-6964b5c0d0ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:39.743989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602a631-1fcb-4693-b59d-6964b5c0d0ee admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:39.840317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1199/2355] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:39 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:39.848677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c37dd-4eee-41e2-ac23-9bf6e30c4efb admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:39.865731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c37dd-4eee-41e2-ac23-9bf6e30c4efb admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:39.974051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52083a66-aab0-40f8-8e01-9b69945fb65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:39.980821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52083a66-aab0-40f8-8e01-9b69945fb65f admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:39.980821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52083a66-aab0-40f8-8e01-9b69945fb65f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:39.980821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52083a66-aab0-40f8-8e01-9b69945fb65f admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:39.980821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52083a66-aab0-40f8-8e01-9b69945fb65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:39.980821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52083a66-aab0-40f8-8e01-9b69945fb65f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:40.076096 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1157/2356] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:39 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:40.077492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602a631-1fcb-4693-b59d-6964b5c0d0ee admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:40.104880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602a631-1fcb-4693-b59d-6964b5c0d0ee admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:40.112514 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62657d-4822-40d9-8286-498de47a31dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:40.114943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62657d-4822-40d9-8286-498de47a31dc admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:40.115722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62657d-4822-40d9-8286-498de47a31dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:40.116187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62657d-4822-40d9-8286-498de47a31dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:40.117642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62657d-4822-40d9-8286-498de47a31dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:40.120086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62657d-4822-40d9-8286-498de47a31dc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:40.133148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1200/2357] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:40.138670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52083a66-aab0-40f8-8e01-9b69945fb65f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:40.152372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52083a66-aab0-40f8-8e01-9b69945fb65f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:40.169935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9cff9-d87f-418c-a6b2-e2eb95cfda7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:40.171287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9cff9-d87f-418c-a6b2-e2eb95cfda7a admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:40.171580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9cff9-d87f-418c-a6b2-e2eb95cfda7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:40.173580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9cff9-d87f-418c-a6b2-e2eb95cfda7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:40.173580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9cff9-d87f-418c-a6b2-e2eb95cfda7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:40.174991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19058979-deb9-4893-ac23-a68b5bca3b7e None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:40.175172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9cff9-d87f-418c-a6b2-e2eb95cfda7a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:40.175656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19058979-deb9-4893-ac23-a68b5bca3b7e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:40.175943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19058979-deb9-4893-ac23-a68b5bca3b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:40.190729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1158/2358] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:40.194584 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9cff9-d87f-418c-a6b2-e2eb95cfda7a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:40.211403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-19058979-deb9-4893-ac23-a68b5bca3b7e None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:40.214186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-19058979-deb9-4893-ac23-a68b5bca3b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:40.220314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9cff9-d87f-418c-a6b2-e2eb95cfda7a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:40.228710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c031f-ae85-484e-af5c-4d28e02377f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:40.228710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c031f-ae85-484e-af5c-4d28e02377f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:40.228710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c031f-ae85-484e-af5c-4d28e02377f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:40.228710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c031f-ae85-484e-af5c-4d28e02377f7 admin admin] PATH_INFO: `/v3/projects/757b69afcbc64d9a853a7ae1f7dc3307/users/7ed43701d36b4b479a2a3a978d242625/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:40.253865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c031f-ae85-484e-af5c-4d28e02377f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=757b69afcbc64d9a853a7ae1f7dc3307, user_id=7ed43701d36b4b479a2a3a978d242625, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:40.256940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c031f-ae85-484e-af5c-4d28e02377f7 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:40.336382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1159/2359] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:40 2022] PUT /identity/v3/projects/757b69afcbc64d9a853a7ae1f7dc3307/users/7ed43701d36b4b479a2a3a978d242625/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:03:40.341779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c031f-ae85-484e-af5c-4d28e02377f7 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:40.344684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e82c031f-ae85-484e-af5c-4d28e02377f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:40.354701 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1201/2360] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:03:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:03:40.357110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4d83b-9362-4d2f-8fec-2a880c4ae718 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:40.358870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4d83b-9362-4d2f-8fec-2a880c4ae718 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:40.359389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4d83b-9362-4d2f-8fec-2a880c4ae718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:40.398984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f3d4d83b-9362-4d2f-8fec-2a880c4ae718 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:40.400308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f3d4d83b-9362-4d2f-8fec-2a880c4ae718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:40.490104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1202/2361] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:40.494299 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4d83b-9362-4d2f-8fec-2a880c4ae718 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:40.495918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f3d4d83b-9362-4d2f-8fec-2a880c4ae718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:40.510331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c031f-ae85-484e-af5c-4d28e02377f7 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:40.520710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:40.522036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:40.522543 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:40.524499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] PATH_INFO: `/v3/users/18255f3f84da457a9bd3db414a49a84f` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:40.527642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18255f3f84da457a9bd3db414a49a84f)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:40.531907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:40.541682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:40.630558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4d83b-9362-4d2f-8fec-2a880c4ae718 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:40.647757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034048b5-a518-4529-a3d9-bee2d3809f93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:40.649446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034048b5-a518-4529-a3d9-bee2d3809f93 service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:40.652081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034048b5-a518-4529-a3d9-bee2d3809f93 service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:40.652503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034048b5-a518-4529-a3d9-bee2d3809f93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:40.753812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034048b5-a518-4529-a3d9-bee2d3809f93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:40.755538 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034048b5-a518-4529-a3d9-bee2d3809f93 service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:40.765652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1203/2362] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:03:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:03:40.766501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034048b5-a518-4529-a3d9-bee2d3809f93 service swift] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:40.768565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-034048b5-a518-4529-a3d9-bee2d3809f93 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:40.814106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:40.842327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18255f3f84da457a9bd3db414a49a84f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:40.861013 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18255f3f84da457a9bd3db414a49a84f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:40.867956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18255f3f84da457a9bd3db414a49a84f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:40.880778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034048b5-a518-4529-a3d9-bee2d3809f93 service swift] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:40.891169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb9c25-687a-40ab-8137-69cea9806d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:40.892694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9c25-687a-40ab-8137-69cea9806d79 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:40.893096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9c25-687a-40ab-8137-69cea9806d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:40.893961 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9c25-687a-40ab-8137-69cea9806d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:40.914341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb9c25-687a-40ab-8137-69cea9806d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:40.916407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb9c25-687a-40ab-8137-69cea9806d79 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:40.929225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1204/2363] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:40.991095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18255f3f84da457a9bd3db414a49a84f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:41.013182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18255f3f84da457a9bd3db414a49a84f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:41.020621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1160/2364] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:40 2022] DELETE /identity/v3/users/18255f3f84da457a9bd3db414a49a84f => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:41.025886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ed8b1-94d5-451b-9901-bd78dcfe8df5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:41.026400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ed8b1-94d5-451b-9901-bd78dcfe8df5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:41.026712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ed8b1-94d5-451b-9901-bd78dcfe8df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:41.047577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-da4ed8b1-94d5-451b-9901-bd78dcfe8df5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:41.055326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-da4ed8b1-94d5-451b-9901-bd78dcfe8df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:41.134099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1205/2365] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:41.151944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:41.154432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:41.324918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa144d3-7b72-407b-aaff-e8a92ba6e591 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:41.336848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ced1a4-a418-43b1-8d6b-68bd097fe38a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:41.337879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ced1a4-a418-43b1-8d6b-68bd097fe38a service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:41.338345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ced1a4-a418-43b1-8d6b-68bd097fe38a service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:41.338786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ced1a4-a418-43b1-8d6b-68bd097fe38a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:41.373698 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ced1a4-a418-43b1-8d6b-68bd097fe38a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:41.375311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ced1a4-a418-43b1-8d6b-68bd097fe38a service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:41.389521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1161/2366] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:41.897292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ed8b1-94d5-451b-9901-bd78dcfe8df5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:41.899778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-da4ed8b1-94d5-451b-9901-bd78dcfe8df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:41.922076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ed8b1-94d5-451b-9901-bd78dcfe8df5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:41.948274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:41.949502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:41.949821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:41.950146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] PATH_INFO: `/v3/projects/d26f18f9c1b442c1b6d03dd3f4982bd3` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:41.963947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d26f18f9c1b442c1b6d03dd3f4982bd3)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:41.965951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:41.994400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:42.136118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd26f18f9c1b442c1b6d03dd3f4982bd3', 'request_id': 'req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:42.151891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd26f18f9c1b442c1b6d03dd3f4982bd3', 'request_id': 'req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:42.161418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd26f18f9c1b442c1b6d03dd3f4982bd3', 'request_id': 'req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:42.934048 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] The token cache is being invalidate because project d26f18f9c1b442c1b6d03dd3f4982bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:42.937424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:42.937424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:42.937424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1206/2367] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:41 2022] DELETE /identity/v3/projects/d26f18f9c1b442c1b6d03dd3f4982bd3 => generated 0 bytes in 1040 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:42.940590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ced1a4-a418-43b1-8d6b-68bd097fe38a service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:42.950353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ced1a4-a418-43b1-8d6b-68bd097fe38a service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:43.011050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:43.011926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:43.012367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:43.012667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] PATH_INFO: `/v3/users/4cf17b0dd2fc4e0e8749dc939d73a620` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:43.013896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf17b0dd2fc4e0e8749dc939d73a620)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:43.016117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:43.309591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf17b0dd2fc4e0e8749dc939d73a620'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:43.320889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf17b0dd2fc4e0e8749dc939d73a620'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:43.321443 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf17b0dd2fc4e0e8749dc939d73a620'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:43.325354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:43.402796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf17b0dd2fc4e0e8749dc939d73a620'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:43.428057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d24a6c40-cdf0-4a91-9bc6-7dab119199f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf17b0dd2fc4e0e8749dc939d73a620'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:43.451513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1162/2368] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:42 2022] DELETE /identity/v3/users/4cf17b0dd2fc4e0e8749dc939d73a620 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:44.367877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:44.370595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:44.424535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda0876-1b35-4a7d-a826-86bc1c4ae3e3 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:44.610404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:44.611587 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:44.612051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:44.612505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] PATH_INFO: `/v3/projects/5d2dd768007e4dab983ec519022dd7e0` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:44.615640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d2dd768007e4dab983ec519022dd7e0)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:44.617933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:44.783908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d2dd768007e4dab983ec519022dd7e0', 'request_id': 'req-31511d53-630c-4f9b-ae70-8953290ab431'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:44.808832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d2dd768007e4dab983ec519022dd7e0', 'request_id': 'req-31511d53-630c-4f9b-ae70-8953290ab431'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:44.825827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d2dd768007e4dab983ec519022dd7e0', 'request_id': 'req-31511d53-630c-4f9b-ae70-8953290ab431'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:44.919919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] The token cache is being invalidate because project 5d2dd768007e4dab983ec519022dd7e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:44.921896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:44.922628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-31511d53-630c-4f9b-ae70-8953290ab431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:44.925870 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1207/2369] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:44 2022] DELETE /identity/v3/projects/5d2dd768007e4dab983ec519022dd7e0 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:44.933237 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b73d-c72a-4422-8ccc-f4a3e0083f9a None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:44.933690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b73d-c72a-4422-8ccc-f4a3e0083f9a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:44.936207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b73d-c72a-4422-8ccc-f4a3e0083f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:44.982028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a0f8b73d-c72a-4422-8ccc-f4a3e0083f9a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:44.982831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a0f8b73d-c72a-4422-8ccc-f4a3e0083f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:45.102175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1163/2370] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:45.110453 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29474e-37ce-47c5-b1f8-be7caaed75aa None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:45.110866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29474e-37ce-47c5-b1f8-be7caaed75aa None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:45.111244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29474e-37ce-47c5-b1f8-be7caaed75aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:45.166936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1b29474e-37ce-47c5-b1f8-be7caaed75aa None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:45.174543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1b29474e-37ce-47c5-b1f8-be7caaed75aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:45.195564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1208/2371] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:45.196345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b73d-c72a-4422-8ccc-f4a3e0083f9a None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:45.214983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b73d-c72a-4422-8ccc-f4a3e0083f9a None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:45.236391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad0147-0b19-43c3-9228-cc5c0b374a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:45.237213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad0147-0b19-43c3-9228-cc5c0b374a86 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:45.237624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad0147-0b19-43c3-9228-cc5c0b374a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:45.237997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad0147-0b19-43c3-9228-cc5c0b374a86 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:45.238848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad0147-0b19-43c3-9228-cc5c0b374a86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:45.241680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad0147-0b19-43c3-9228-cc5c0b374a86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:45.245622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad0147-0b19-43c3-9228-cc5c0b374a86 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:45.270560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1164/2372] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:03:45.276829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29474e-37ce-47c5-b1f8-be7caaed75aa None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:45.302583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29474e-37ce-47c5-b1f8-be7caaed75aa None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:45.313980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94856084-9690-4c0d-888b-d328ee8abb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:45.314671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94856084-9690-4c0d-888b-d328ee8abb95 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:45.315001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94856084-9690-4c0d-888b-d328ee8abb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:45.315265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94856084-9690-4c0d-888b-d328ee8abb95 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:45.315969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94856084-9690-4c0d-888b-d328ee8abb95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:45.319235 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94856084-9690-4c0d-888b-d328ee8abb95 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:45.454859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:45.467645 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1209/2373] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:45 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:45.472367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad0147-0b19-43c3-9228-cc5c0b374a86 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:45.505842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad0147-0b19-43c3-9228-cc5c0b374a86 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:45.665421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc459f3c-3ab6-4065-9e3b-58f687e61a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:45.666388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459f3c-3ab6-4065-9e3b-58f687e61a2d admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:45.666769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459f3c-3ab6-4065-9e3b-58f687e61a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:45.667354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459f3c-3ab6-4065-9e3b-58f687e61a2d admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:45.668117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc459f3c-3ab6-4065-9e3b-58f687e61a2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:45.670245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc459f3c-3ab6-4065-9e3b-58f687e61a2d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:45.798185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1165/2374] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:45 2022] POST /identity/v3/users => generated 287 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:45.802932 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94856084-9690-4c0d-888b-d328ee8abb95 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:45.816918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94856084-9690-4c0d-888b-d328ee8abb95 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:45.822600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dba267-3173-4b20-8b3f-abc188a2293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:45.823307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dba267-3173-4b20-8b3f-abc188a2293c admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:45.825405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dba267-3173-4b20-8b3f-abc188a2293c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:45.825405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dba267-3173-4b20-8b3f-abc188a2293c admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:45.825405 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dba267-3173-4b20-8b3f-abc188a2293c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:45.826619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dba267-3173-4b20-8b3f-abc188a2293c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:45.839081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1210/2375] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:45.843012 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc459f3c-3ab6-4065-9e3b-58f687e61a2d admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:45.853576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc459f3c-3ab6-4065-9e3b-58f687e61a2d admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:45.858621 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50153b9b-0f35-4c19-bbf2-1e39b99e940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:45.859378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50153b9b-0f35-4c19-bbf2-1e39b99e940e admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:45.859650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50153b9b-0f35-4c19-bbf2-1e39b99e940e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:45.860228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50153b9b-0f35-4c19-bbf2-1e39b99e940e admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:45.860927 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50153b9b-0f35-4c19-bbf2-1e39b99e940e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:45.862564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50153b9b-0f35-4c19-bbf2-1e39b99e940e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:45.877317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1166/2376] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:45.878787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dba267-3173-4b20-8b3f-abc188a2293c admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:45.899581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dba267-3173-4b20-8b3f-abc188a2293c admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:45.907492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bc628-a368-40f5-8382-0035f386f613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:45.908409 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bc628-a368-40f5-8382-0035f386f613 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:45.908722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bc628-a368-40f5-8382-0035f386f613 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:45.909001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bc628-a368-40f5-8382-0035f386f613 admin admin] PATH_INFO: `/v3/projects/22e598739ae64ebf9e523e4e2e6a2867/users/46a641c6357148f984bd21afe6cfb242/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:45.926977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bc628-a368-40f5-8382-0035f386f613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e598739ae64ebf9e523e4e2e6a2867, user_id=46a641c6357148f984bd21afe6cfb242, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:45.929744 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bc628-a368-40f5-8382-0035f386f613 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:46.073935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1211/2377] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:45 2022] PUT /identity/v3/projects/22e598739ae64ebf9e523e4e2e6a2867/users/46a641c6357148f984bd21afe6cfb242/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:03:46.078759 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83a667-c441-4a06-8d1a-fe439dff3e3d None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:46.080135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83a667-c441-4a06-8d1a-fe439dff3e3d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:46.080591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83a667-c441-4a06-8d1a-fe439dff3e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:46.110873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3a83a667-c441-4a06-8d1a-fe439dff3e3d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:46.117181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3a83a667-c441-4a06-8d1a-fe439dff3e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:46.202581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:46.274441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1167/2378] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:46.285328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bc628-a368-40f5-8382-0035f386f613 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:46.287609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4b6bc628-a368-40f5-8382-0035f386f613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:46.444705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bc628-a368-40f5-8382-0035f386f613 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:46.452607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd41d3a-3807-45c7-81f2-af2b6ef6897a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:46.453693 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd41d3a-3807-45c7-81f2-af2b6ef6897a service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:46.454181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd41d3a-3807-45c7-81f2-af2b6ef6897a service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:46.454621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd41d3a-3807-45c7-81f2-af2b6ef6897a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:46.484546 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd41d3a-3807-45c7-81f2-af2b6ef6897a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:46.486318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd41d3a-3807-45c7-81f2-af2b6ef6897a service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:46.501693 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1212/2379] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:49.659433 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d98b4-4e19-4719-bdc0-a64cd9c27e5b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:49.660601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d98b4-4e19-4719-bdc0-a64cd9c27e5b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:49.661184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d98b4-4e19-4719-bdc0-a64cd9c27e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:49.765091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b41d98b4-4e19-4719-bdc0-a64cd9c27e5b None None] MFA Rules not processed for user `46a641c6357148f984bd21afe6cfb242`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:49.765091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b41d98b4-4e19-4719-bdc0-a64cd9c27e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:49.827976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1168/2380] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:49.839320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd41d3a-3807-45c7-81f2-af2b6ef6897a service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:49.840894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5fd41d3a-3807-45c7-81f2-af2b6ef6897a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:49.921565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd41d3a-3807-45c7-81f2-af2b6ef6897a service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:49.926887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6b4aa-8f69-4f3a-b195-79e32eb53f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:49.928001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6b4aa-8f69-4f3a-b195-79e32eb53f52 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:49.928467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6b4aa-8f69-4f3a-b195-79e32eb53f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:49.928963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6b4aa-8f69-4f3a-b195-79e32eb53f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:49.941114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6b4aa-8f69-4f3a-b195-79e32eb53f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:49.943123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6b4aa-8f69-4f3a-b195-79e32eb53f52 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:49.966395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1213/2381] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:03:52.841182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d98b4-4e19-4719-bdc0-a64cd9c27e5b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:52.845752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b41d98b4-4e19-4719-bdc0-a64cd9c27e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:52.886021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d98b4-4e19-4719-bdc0-a64cd9c27e5b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:52.911234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:52.917220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:52.917545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:52.917834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] PATH_INFO: `/v3/users/b43cbcee8821452897e173c6e0c77a82` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:52.926321 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b43cbcee8821452897e173c6e0c77a82)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:52.938794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:53.120251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d93ab-19ef-428b-a192-c1ab8b82b378 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:53.121226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d93ab-19ef-428b-a192-c1ab8b82b378 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:53.121226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d93ab-19ef-428b-a192-c1ab8b82b378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:53.197452 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-0b0d93ab-19ef-428b-a192-c1ab8b82b378 None None] MFA Rules not processed for user `7ed43701d36b4b479a2a3a978d242625`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:53.200062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0b0d93ab-19ef-428b-a192-c1ab8b82b378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:53.363426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:53.375957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1214/2382] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:53.377655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984d84f-b358-4724-bdbe-515ac11179f7 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:53.378024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984d84f-b358-4724-bdbe-515ac11179f7 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:53.378383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984d84f-b358-4724-bdbe-515ac11179f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:53.414747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c984d84f-b358-4724-bdbe-515ac11179f7 None None] MFA Rules not processed for user `46a641c6357148f984bd21afe6cfb242`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:53.418253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c984d84f-b358-4724-bdbe-515ac11179f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:53.435283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1215/2383] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:53.439255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984d84f-b358-4724-bdbe-515ac11179f7 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:53.440864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c984d84f-b358-4724-bdbe-515ac11179f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:53.457255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984d84f-b358-4724-bdbe-515ac11179f7 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:53.463612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e286c1-ac9c-4808-bd46-84a58fe0589d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:53.464978 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e286c1-ac9c-4808-bd46-84a58fe0589d admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:53.465919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e286c1-ac9c-4808-bd46-84a58fe0589d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:53.466434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e286c1-ac9c-4808-bd46-84a58fe0589d admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:53.467390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e286c1-ac9c-4808-bd46-84a58fe0589d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:53.470149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e286c1-ac9c-4808-bd46-84a58fe0589d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:53.502189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b43cbcee8821452897e173c6e0c77a82'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:53.539090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b43cbcee8821452897e173c6e0c77a82'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:53.539090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b43cbcee8821452897e173c6e0c77a82'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:53.627679 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1216/2384] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:53 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:53.628907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e286c1-ac9c-4808-bd46-84a58fe0589d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:53.637573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b43cbcee8821452897e173c6e0c77a82'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:53.711917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-3c07021a-3c6e-44cf-b81a-b095698ac9f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b43cbcee8821452897e173c6e0c77a82'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:53.747037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1169/2385] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:03:52 2022] DELETE /identity/v3/users/b43cbcee8821452897e173c6e0c77a82 => generated 0 bytes in 900 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:53.748252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71c8ea-4d6a-454f-9a73-ff6658cce43f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:53.748911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71c8ea-4d6a-454f-9a73-ff6658cce43f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:53.749462 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71c8ea-4d6a-454f-9a73-ff6658cce43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:53.805097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-7c71c8ea-4d6a-454f-9a73-ff6658cce43f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:53.811173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7c71c8ea-4d6a-454f-9a73-ff6658cce43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:53.832893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e286c1-ac9c-4808-bd46-84a58fe0589d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:54.009873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:54.022987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f699c-f934-480e-9769-57283eb3ca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:54.023929 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f699c-f934-480e-9769-57283eb3ca86 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:54.024216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f699c-f934-480e-9769-57283eb3ca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:54.024380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f699c-f934-480e-9769-57283eb3ca86 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:54.025139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f699c-f934-480e-9769-57283eb3ca86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:54.027387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f699c-f934-480e-9769-57283eb3ca86 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:54.089867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1170/2386] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:54.106670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71c8ea-4d6a-454f-9a73-ff6658cce43f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:54.111961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7c71c8ea-4d6a-454f-9a73-ff6658cce43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:54.263324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1217/2387] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:53 2022] POST /identity/v3/users => generated 295 bytes in 635 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:54.268088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f699c-f934-480e-9769-57283eb3ca86 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:54.269835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-bd3f699c-f934-480e-9769-57283eb3ca86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:54.281162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71c8ea-4d6a-454f-9a73-ff6658cce43f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:54.292804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751a37-4757-4dfa-9306-0548421b6561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:54.299549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751a37-4757-4dfa-9306-0548421b6561 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:54.306843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751a37-4757-4dfa-9306-0548421b6561 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:54.306843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751a37-4757-4dfa-9306-0548421b6561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:54.308208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f699c-f934-480e-9769-57283eb3ca86 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:54.317400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc79036-2a4a-4a57-b73a-2e26cc9e2843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:54.318110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc79036-2a4a-4a57-b73a-2e26cc9e2843 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:54.318404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc79036-2a4a-4a57-b73a-2e26cc9e2843 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:54.318823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc79036-2a4a-4a57-b73a-2e26cc9e2843 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:54.319448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc79036-2a4a-4a57-b73a-2e26cc9e2843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:54.326402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc79036-2a4a-4a57-b73a-2e26cc9e2843 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:54.335873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751a37-4757-4dfa-9306-0548421b6561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:54.335873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751a37-4757-4dfa-9306-0548421b6561 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:54.358034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1218/2388] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:54.364445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc79036-2a4a-4a57-b73a-2e26cc9e2843 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:54.383007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1171/2389] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:03:54.407373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc79036-2a4a-4a57-b73a-2e26cc9e2843 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:54.431903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb24770-8031-4d55-afe2-9a03c0afe176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:54.433808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb24770-8031-4d55-afe2-9a03c0afe176 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:54.436566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb24770-8031-4d55-afe2-9a03c0afe176 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:54.437775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb24770-8031-4d55-afe2-9a03c0afe176 admin admin] PATH_INFO: `/v3/projects/ffb48bb456c8458483f20ab4e2a0462c/users/fb20ceed686b468bbff462e7272b29cb/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:54.709957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb24770-8031-4d55-afe2-9a03c0afe176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb48bb456c8458483f20ab4e2a0462c, user_id=fb20ceed686b468bbff462e7272b29cb, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:54.709957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb24770-8031-4d55-afe2-9a03c0afe176 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:54.896528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1219/2390] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:54 2022] PUT /identity/v3/projects/ffb48bb456c8458483f20ab4e2a0462c/users/fb20ceed686b468bbff462e7272b29cb/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:03:54.898167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751a37-4757-4dfa-9306-0548421b6561 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:54.916130 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751a37-4757-4dfa-9306-0548421b6561 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:55.099422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af60b6-4944-479f-bda3-3f73e355082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:55.100260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af60b6-4944-479f-bda3-3f73e355082f admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:55.100627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af60b6-4944-479f-bda3-3f73e355082f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:55.101075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af60b6-4944-479f-bda3-3f73e355082f admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:55.102112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af60b6-4944-479f-bda3-3f73e355082f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:55.112432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af60b6-4944-479f-bda3-3f73e355082f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:55.158010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1172/2391] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:55.162543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb24770-8031-4d55-afe2-9a03c0afe176 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:55.191732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb24770-8031-4d55-afe2-9a03c0afe176 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:55.218010 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e77b5c-b9cc-4863-89b4-e1bf3abc1501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:55.219061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e77b5c-b9cc-4863-89b4-e1bf3abc1501 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:55.219403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e77b5c-b9cc-4863-89b4-e1bf3abc1501 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:55.224810 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e77b5c-b9cc-4863-89b4-e1bf3abc1501 admin admin] PATH_INFO: `/v3/projects/ffb48bb456c8458483f20ab4e2a0462c/users/fb20ceed686b468bbff462e7272b29cb/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:55.227795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e77b5c-b9cc-4863-89b4-e1bf3abc1501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb48bb456c8458483f20ab4e2a0462c, user_id=fb20ceed686b468bbff462e7272b29cb, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:55.230673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e77b5c-b9cc-4863-89b4-e1bf3abc1501 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:55.260860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1220/2392] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:55 2022] PUT /identity/v3/projects/ffb48bb456c8458483f20ab4e2a0462c/users/fb20ceed686b468bbff462e7272b29cb/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:55.687798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af60b6-4944-479f-bda3-3f73e355082f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:55.693521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f2af60b6-4944-479f-bda3-3f73e355082f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:55.762021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af60b6-4944-479f-bda3-3f73e355082f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:55.928227 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:55.930847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:55.931578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:55.932017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] PATH_INFO: `/v3/projects/80265852e5994f3689366da21c99be5b` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:55.941058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] RBAC: Authorizing `identity:delete_project(project_id=80265852e5994f3689366da21c99be5b)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:55.946619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:56.285392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:56.311983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80265852e5994f3689366da21c99be5b', 'request_id': 'req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:56.320987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80265852e5994f3689366da21c99be5b', 'request_id': 'req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:56.426365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80265852e5994f3689366da21c99be5b', 'request_id': 'req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:56.441372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] The token cache is being invalidate because project 80265852e5994f3689366da21c99be5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:03:56.441693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:56.442619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-4a178ccd-41ea-444c-b9b4-fb8ec4cf166f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:03:56.448155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1173/2393] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:03:55 2022] DELETE /identity/v3/projects/80265852e5994f3689366da21c99be5b => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:56.454485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a295d1c-5d3a-49a2-93bf-dfddaacbafa2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:56.455088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a295d1c-5d3a-49a2-93bf-dfddaacbafa2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:56.455088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a295d1c-5d3a-49a2-93bf-dfddaacbafa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:56.492464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-7a295d1c-5d3a-49a2-93bf-dfddaacbafa2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:56.493813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7a295d1c-5d3a-49a2-93bf-dfddaacbafa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:56.560151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:03:56.666875 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c426dd-ed2c-469d-9227-65a7eed7d944 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:56.667545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c426dd-ed2c-469d-9227-65a7eed7d944 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:56.668127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c426dd-ed2c-469d-9227-65a7eed7d944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:56.668389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1221/2394] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:56.719973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c1c426dd-ed2c-469d-9227-65a7eed7d944 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:56.721718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c1c426dd-ed2c-469d-9227-65a7eed7d944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:56.744981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1174/2395] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:56.750483 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a295d1c-5d3a-49a2-93bf-dfddaacbafa2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:56.763201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a295d1c-5d3a-49a2-93bf-dfddaacbafa2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:56.771554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092de0be-b57f-4f8e-a5cb-9650c88f92af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:56.772540 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092de0be-b57f-4f8e-a5cb-9650c88f92af admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:56.773051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092de0be-b57f-4f8e-a5cb-9650c88f92af admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:56.773326 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092de0be-b57f-4f8e-a5cb-9650c88f92af admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:56.774259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092de0be-b57f-4f8e-a5cb-9650c88f92af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:03:56.774736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092de0be-b57f-4f8e-a5cb-9650c88f92af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:56.776604 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092de0be-b57f-4f8e-a5cb-9650c88f92af admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:56.796704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1222/2396] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:03:56.797272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c426dd-ed2c-469d-9227-65a7eed7d944 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:56.810847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c426dd-ed2c-469d-9227-65a7eed7d944 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:56.836870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf8dd4-2594-4321-a890-8faf67c01724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:56.839040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf8dd4-2594-4321-a890-8faf67c01724 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:56.843575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf8dd4-2594-4321-a890-8faf67c01724 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:56.844465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf8dd4-2594-4321-a890-8faf67c01724 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:56.846215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf8dd4-2594-4321-a890-8faf67c01724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:56.856412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf8dd4-2594-4321-a890-8faf67c01724 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:56.984189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1175/2397] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:03:56 2022] POST /identity/v3/projects => generated 399 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:03:56.988256 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092de0be-b57f-4f8e-a5cb-9650c88f92af admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:57.007518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092de0be-b57f-4f8e-a5cb-9650c88f92af admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:57.123170 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a3038-b394-40e4-b230-7a2b21e2c903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:57.123843 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a3038-b394-40e4-b230-7a2b21e2c903 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:57.125759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a3038-b394-40e4-b230-7a2b21e2c903 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:57.126284 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a3038-b394-40e4-b230-7a2b21e2c903 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:57.129447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a3038-b394-40e4-b230-7a2b21e2c903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:57.133557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a3038-b394-40e4-b230-7a2b21e2c903 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:57.287251 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1223/2398] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:03:56 2022] POST /identity/v3/users => generated 300 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:03:57.289811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf8dd4-2594-4321-a890-8faf67c01724 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:57.321568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf8dd4-2594-4321-a890-8faf67c01724 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:57.327924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec0b88-ce46-43b4-a9fa-20d64cdfc9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:57.329072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec0b88-ce46-43b4-a9fa-20d64cdfc9ab admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:57.329675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec0b88-ce46-43b4-a9fa-20d64cdfc9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:57.330177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec0b88-ce46-43b4-a9fa-20d64cdfc9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:57.331194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec0b88-ce46-43b4-a9fa-20d64cdfc9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:57.334381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec0b88-ce46-43b4-a9fa-20d64cdfc9ab admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:57.365711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1176/2399] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:57.371925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a3038-b394-40e4-b230-7a2b21e2c903 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:57.382771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a3038-b394-40e4-b230-7a2b21e2c903 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:57.387775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47cadc-f52f-41fd-930e-ed1b333af8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:57.388919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47cadc-f52f-41fd-930e-ed1b333af8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:57.388919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47cadc-f52f-41fd-930e-ed1b333af8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:57.389128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47cadc-f52f-41fd-930e-ed1b333af8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:57.389996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47cadc-f52f-41fd-930e-ed1b333af8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:57.391641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47cadc-f52f-41fd-930e-ed1b333af8b7 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:57.420763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1224/2400] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:03:57.423194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec0b88-ce46-43b4-a9fa-20d64cdfc9ab admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:57.447485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec0b88-ce46-43b4-a9fa-20d64cdfc9ab admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:57.490113 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a88357-963f-41ab-a110-2dba7d8ca5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:57.494320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88357-963f-41ab-a110-2dba7d8ca5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:57.500035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88357-963f-41ab-a110-2dba7d8ca5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:57.500252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88357-963f-41ab-a110-2dba7d8ca5fb admin admin] PATH_INFO: `/v3/projects/845c1597e02047eb9625ad84c42b43cb/users/e1c7d4f2b4dc43ceac55b5d524e07c11/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:57.542991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a88357-963f-41ab-a110-2dba7d8ca5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=845c1597e02047eb9625ad84c42b43cb, user_id=e1c7d4f2b4dc43ceac55b5d524e07c11, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:57.547464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a88357-963f-41ab-a110-2dba7d8ca5fb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:57.672188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1177/2401] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:03:57 2022] PUT /identity/v3/projects/845c1597e02047eb9625ad84c42b43cb/users/e1c7d4f2b4dc43ceac55b5d524e07c11/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:03:57.687556 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efaccb-f795-4e9c-b799-b465ad070428 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:57.688750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efaccb-f795-4e9c-b799-b465ad070428 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:57.689354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efaccb-f795-4e9c-b799-b465ad070428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:57.751739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-14efaccb-f795-4e9c-b799-b465ad070428 None None] MFA Rules not processed for user `e1c7d4f2b4dc43ceac55b5d524e07c11`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:03:57.754595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-14efaccb-f795-4e9c-b799-b465ad070428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:57.924885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1225/2402] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:03:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:03:57.935188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a88357-963f-41ab-a110-2dba7d8ca5fb admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:03:57.938411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c0a88357-963f-41ab-a110-2dba7d8ca5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:03:58.019887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a88357-963f-41ab-a110-2dba7d8ca5fb admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:03:58.030686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af5653-a5e5-4f90-a61f-b1cb6749332f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:03:58.031677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5653-a5e5-4f90-a61f-b1cb6749332f service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:03:58.032931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5653-a5e5-4f90-a61f-b1cb6749332f service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:03:58.032931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5653-a5e5-4f90-a61f-b1cb6749332f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:03:58.046767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af5653-a5e5-4f90-a61f-b1cb6749332f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:03:58.050374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af5653-a5e5-4f90-a61f-b1cb6749332f service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:03:58.086873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1178/2403] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:03:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:04:02.807141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e704ed1-f96b-4b66-95da-47b021242eb7 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:02.807700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e704ed1-f96b-4b66-95da-47b021242eb7 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:02.807804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e704ed1-f96b-4b66-95da-47b021242eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:02.836114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:02.841397 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-5e704ed1-f96b-4b66-95da-47b021242eb7 None None] MFA Rules not processed for user `fb20ceed686b468bbff462e7272b29cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:02.842846 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5e704ed1-f96b-4b66-95da-47b021242eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:02.988097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1226/2404] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:04:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:03.045731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af5653-a5e5-4f90-a61f-b1cb6749332f service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:03.047161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-78af5653-a5e5-4f90-a61f-b1cb6749332f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:03.139084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:03.184346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af5653-a5e5-4f90-a61f-b1cb6749332f service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:03.191816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7d471-02df-4ce7-9672-515919162a03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:03.193242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7d471-02df-4ce7-9672-515919162a03 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:03.193542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7d471-02df-4ce7-9672-515919162a03 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:03.193828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7d471-02df-4ce7-9672-515919162a03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:03.201554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9fc7d471-02df-4ce7-9672-515919162a03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:03.296422 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7d471-02df-4ce7-9672-515919162a03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:03.299252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7d471-02df-4ce7-9672-515919162a03 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:03.314845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1179/2405] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:04:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:04:04.795173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd03f7-772f-4d1b-95f6-0b733c58a4f3 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:04.795730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd03f7-772f-4d1b-95f6-0b733c58a4f3 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:04.795730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd03f7-772f-4d1b-95f6-0b733c58a4f3 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:04.797637 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1227/2406] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:04:04.804072 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe1658-c307-41e0-9390-190caaaf37bb None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:04.805006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe1658-c307-41e0-9390-190caaaf37bb None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:04.805727 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe1658-c307-41e0-9390-190caaaf37bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:04.827831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-8cfe1658-c307-41e0-9390-190caaaf37bb None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:04.829175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8cfe1658-c307-41e0-9390-190caaaf37bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:04.839523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1180/2407] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:04:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:04:04.852334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd03f7-772f-4d1b-95f6-0b733c58a4f3 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:04.854722 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-42cd03f7-772f-4d1b-95f6-0b733c58a4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:04.965201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd03f7-772f-4d1b-95f6-0b733c58a4f3 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:04.969687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30249d88-9f4e-42e0-82e2-012c5b56d242 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca33ef49ba44832a7fb45b2317d211c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:04.970352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30249d88-9f4e-42e0-82e2-012c5b56d242 service glance] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:04.970644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30249d88-9f4e-42e0-82e2-012c5b56d242 service glance] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:04.970924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30249d88-9f4e-42e0-82e2-012c5b56d242 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:04.995628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30249d88-9f4e-42e0-82e2-012c5b56d242 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:04.998439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30249d88-9f4e-42e0-82e2-012c5b56d242 service glance] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:05.015737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1228/2408] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:04:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:04:05.176520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb831d8-87c7-46d1-b973-f702bcf455ca None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:05.177385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb831d8-87c7-46d1-b973-f702bcf455ca None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:05.177385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb831d8-87c7-46d1-b973-f702bcf455ca None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:05.178596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1181/2409] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:04:05.184515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f21e61-276f-41dd-9023-c825fff26be1 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:05.184889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f21e61-276f-41dd-9023-c825fff26be1 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:05.185174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f21e61-276f-41dd-9023-c825fff26be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:05.205378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-70f21e61-276f-41dd-9023-c825fff26be1 None None] MFA Rules not processed for user `b9763f9899214df3929f3816f009e857`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:05.206959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-70f21e61-276f-41dd-9023-c825fff26be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:05.268301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1229/2410] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:04:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:05.280188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb831d8-87c7-46d1-b973-f702bcf455ca None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:05.283120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7fb831d8-87c7-46d1-b973-f702bcf455ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:05.310151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb831d8-87c7-46d1-b973-f702bcf455ca None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:05.316977 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea73e1-7421-4aee-9ff9-c0e9f209d014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:05.318643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea73e1-7421-4aee-9ff9-c0e9f209d014 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:05.321434 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea73e1-7421-4aee-9ff9-c0e9f209d014 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:05.321434 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea73e1-7421-4aee-9ff9-c0e9f209d014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:05.334585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea73e1-7421-4aee-9ff9-c0e9f209d014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:05.336467 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea73e1-7421-4aee-9ff9-c0e9f209d014 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:05.362159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1182/2411] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:04:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:04:05.465190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23f6ca-0c62-4752-afe2-6f040e145a7c None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:05.470517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23f6ca-0c62-4752-afe2-6f040e145a7c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:05.470517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23f6ca-0c62-4752-afe2-6f040e145a7c None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:05.470517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1230/2412] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:04:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:04:05.473494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10d2c7-68f3-4bf2-9475-88efd6139c8b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:05.473961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10d2c7-68f3-4bf2-9475-88efd6139c8b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:05.474990 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10d2c7-68f3-4bf2-9475-88efd6139c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:05.499501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-9a10d2c7-68f3-4bf2-9475-88efd6139c8b None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:05.499501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9a10d2c7-68f3-4bf2-9475-88efd6139c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:05.509298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1183/2413] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:04:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:04:05.521729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23f6ca-0c62-4752-afe2-6f040e145a7c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:05.523952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ae23f6ca-0c62-4752-afe2-6f040e145a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:05.538869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23f6ca-0c62-4752-afe2-6f040e145a7c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:05.543292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc06f82-c1d9-4dd9-9890-fa94bb37cce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:05.544026 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc06f82-c1d9-4dd9-9890-fa94bb37cce3 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:05.544524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc06f82-c1d9-4dd9-9890-fa94bb37cce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:05.545044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc06f82-c1d9-4dd9-9890-fa94bb37cce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:05.557926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc06f82-c1d9-4dd9-9890-fa94bb37cce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:05.558783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc06f82-c1d9-4dd9-9890-fa94bb37cce3 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:05.571473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1231/2414] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:04:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:04:10.806471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10d2c7-68f3-4bf2-9475-88efd6139c8b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:10.808566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9a10d2c7-68f3-4bf2-9475-88efd6139c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:10.871032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc06f82-c1d9-4dd9-9890-fa94bb37cce3 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:10.893861 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10d2c7-68f3-4bf2-9475-88efd6139c8b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:10.899254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:10.900148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:10.900490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:10.900794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] PATH_INFO: `/v3/users/77d902e9f51f433880a6ec574a9abfa1` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:10.902012 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77d902e9f51f433880a6ec574a9abfa1)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:10.903961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:10.927301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:10.953394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc06f82-c1d9-4dd9-9890-fa94bb37cce3 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:10.958173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:10.959106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:10.959392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:10.959669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] PATH_INFO: `/v3/users/e1c7d4f2b4dc43ceac55b5d524e07c11` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:10.960902 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1c7d4f2b4dc43ceac55b5d524e07c11)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:10.962866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:11.139421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77d902e9f51f433880a6ec574a9abfa1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.147556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77d902e9f51f433880a6ec574a9abfa1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.152045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77d902e9f51f433880a6ec574a9abfa1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.215208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1c7d4f2b4dc43ceac55b5d524e07c11'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.226774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77d902e9f51f433880a6ec574a9abfa1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.231951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:11.245577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1c7d4f2b4dc43ceac55b5d524e07c11'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.259922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-440f5236-5076-41e3-b90c-fbcb52962639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77d902e9f51f433880a6ec574a9abfa1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.266871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1c7d4f2b4dc43ceac55b5d524e07c11'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.266871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1c7d4f2b4dc43ceac55b5d524e07c11'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.271686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1184/2415] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:04:10 2022] DELETE /identity/v3/users/77d902e9f51f433880a6ec574a9abfa1 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:04:11.273164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5a572-1bd9-4e5e-ac73-6f07f7b98397 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:11.273456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5a572-1bd9-4e5e-ac73-6f07f7b98397 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:11.273799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5a572-1bd9-4e5e-ac73-6f07f7b98397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:11.342836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d2f5a572-1bd9-4e5e-ac73-6f07f7b98397 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:11.344052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d2f5a572-1bd9-4e5e-ac73-6f07f7b98397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:11.391788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1c7d4f2b4dc43ceac55b5d524e07c11'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:11.416506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1185/2417] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:11.422346 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1232/2417] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:04:10 2022] DELETE /identity/v3/users/e1c7d4f2b4dc43ceac55b5d524e07c11 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:04:11.425114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69f99a-7bd3-478e-823a-05c4bc6d6ce0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:11.425925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69f99a-7bd3-478e-823a-05c4bc6d6ce0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:11.426373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69f99a-7bd3-478e-823a-05c4bc6d6ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:11.432566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:11.433694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:11.450098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3e69f99a-7bd3-478e-823a-05c4bc6d6ce0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:11.455380 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3e69f99a-7bd3-478e-823a-05c4bc6d6ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:11.508541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478852b7-3803-42e1-b90a-0d2772d878d2 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:11.513322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3adf1b-6424-4380-b869-0ab009e6e15a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:11.520344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3adf1b-6424-4380-b869-0ab009e6e15a service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:11.520344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3adf1b-6424-4380-b869-0ab009e6e15a service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:11.520344 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3adf1b-6424-4380-b869-0ab009e6e15a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:11.546267 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1186/2418] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:11.557892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69f99a-7bd3-478e-823a-05c4bc6d6ce0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:11.574157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69f99a-7bd3-478e-823a-05c4bc6d6ce0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:11.581024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476a370-be5b-40c6-9f9a-868f9e4504a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:11.582166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476a370-be5b-40c6-9f9a-868f9e4504a5 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:11.582550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476a370-be5b-40c6-9f9a-868f9e4504a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:11.582899 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476a370-be5b-40c6-9f9a-868f9e4504a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:11.595744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476a370-be5b-40c6-9f9a-868f9e4504a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:11.597857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476a370-be5b-40c6-9f9a-868f9e4504a5 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:11.610866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1187/2419] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:04:11.612163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3adf1b-6424-4380-b869-0ab009e6e15a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:11.613803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3adf1b-6424-4380-b869-0ab009e6e15a service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:11.624965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1233/2420] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:04:12.343905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476a370-be5b-40c6-9f9a-868f9e4504a5 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:12.345441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d476a370-be5b-40c6-9f9a-868f9e4504a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:12.453550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3adf1b-6424-4380-b869-0ab009e6e15a service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:12.455226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9c3adf1b-6424-4380-b869-0ab009e6e15a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:12.484393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3adf1b-6424-4380-b869-0ab009e6e15a service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:12.491892 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:12.493329 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:12.493627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:12.494003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] PATH_INFO: `/v3/projects/845c1597e02047eb9625ad84c42b43cb` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:12.498488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=845c1597e02047eb9625ad84c42b43cb)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:12.501052 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:12.506820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476a370-be5b-40c6-9f9a-868f9e4504a5 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:12.507983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:12.538615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:12.540260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:12.540560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:12.540868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:12.546111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:12.587857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:12.590572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:12.598747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:12.605671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1188/2421] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:04:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:04:12.606684 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:12.608313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:12.649827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355ca92-01d1-4bc9-90b1-8e6472d7482a service placement] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:12.664167 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:12.665188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:12.665667 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:12.665954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] PATH_INFO: `/v3/projects/d130b3a0b56e4381a39350b966f5c29a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:12.680266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d130b3a0b56e4381a39350b966f5c29a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:12.683746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:12.723519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '845c1597e02047eb9625ad84c42b43cb', 'request_id': 'req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:12.749853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '845c1597e02047eb9625ad84c42b43cb', 'request_id': 'req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:12.771948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '845c1597e02047eb9625ad84c42b43cb', 'request_id': 'req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:12.859450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] The token cache is being invalidate because project 845c1597e02047eb9625ad84c42b43cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:04:12.860762 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:12.861916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-1c85d5ed-dcf5-49c2-8a14-73ad852bf956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:12.864912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1234/2422] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:04:12 2022] DELETE /identity/v3/projects/845c1597e02047eb9625ad84c42b43cb => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:04:12.930681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd130b3a0b56e4381a39350b966f5c29a', 'request_id': 'req-36e6db42-f919-4821-ab8e-61c93706dfdb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:12.973447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd130b3a0b56e4381a39350b966f5c29a', 'request_id': 'req-36e6db42-f919-4821-ab8e-61c93706dfdb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:13.055475 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd130b3a0b56e4381a39350b966f5c29a', 'request_id': 'req-36e6db42-f919-4821-ab8e-61c93706dfdb'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:13.093147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] The token cache is being invalidate because project d130b3a0b56e4381a39350b966f5c29a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:04:13.093450 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:13.101256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-36e6db42-f919-4821-ab8e-61c93706dfdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:04:13.103644 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1189/2423] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:04:12 2022] DELETE /identity/v3/projects/d130b3a0b56e4381a39350b966f5c29a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:04:13.122167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bbcbe-e29c-4c59-9324-a492c0a6d976 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:13.122512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bbcbe-e29c-4c59-9324-a492c0a6d976 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:13.122749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bbcbe-e29c-4c59-9324-a492c0a6d976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:13.166212 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2c8bbcbe-e29c-4c59-9324-a492c0a6d976 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:13.170285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2c8bbcbe-e29c-4c59-9324-a492c0a6d976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:13.313163 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1235/2424] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:13.319583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fae84-38d8-467e-8c8c-2fb639fd3395 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:13.320892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fae84-38d8-467e-8c8c-2fb639fd3395 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:13.321263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fae84-38d8-467e-8c8c-2fb639fd3395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:13.364049 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-421fae84-38d8-467e-8c8c-2fb639fd3395 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:13.365316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-421fae84-38d8-467e-8c8c-2fb639fd3395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:13.378102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1190/2425] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:13.382778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bbcbe-e29c-4c59-9324-a492c0a6d976 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:13.393644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bbcbe-e29c-4c59-9324-a492c0a6d976 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:13.398678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1815d-0b27-4a51-8f84-5f4fd529d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:13.399763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1815d-0b27-4a51-8f84-5f4fd529d764 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:13.400266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1815d-0b27-4a51-8f84-5f4fd529d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:13.401248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1815d-0b27-4a51-8f84-5f4fd529d764 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:13.402947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1815d-0b27-4a51-8f84-5f4fd529d764 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:04:13.403273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1815d-0b27-4a51-8f84-5f4fd529d764 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:13.406213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1815d-0b27-4a51-8f84-5f4fd529d764 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:13.433004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1236/2426] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:04:13.434970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fae84-38d8-467e-8c8c-2fb639fd3395 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:13.446381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fae84-38d8-467e-8c8c-2fb639fd3395 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:13.452116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e8c1e-1056-405c-a2ea-af38762401fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:13.452836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e8c1e-1056-405c-a2ea-af38762401fc admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:13.453157 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e8c1e-1056-405c-a2ea-af38762401fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:13.453414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e8c1e-1056-405c-a2ea-af38762401fc admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:13.454308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e8c1e-1056-405c-a2ea-af38762401fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:13.456341 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e8c1e-1056-405c-a2ea-af38762401fc admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:13.572036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1191/2427] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:04:13 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:04:13.577201 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1815d-0b27-4a51-8f84-5f4fd529d764 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:13.614217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1815d-0b27-4a51-8f84-5f4fd529d764 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:13.701339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7c92b-4ee2-4a10-8339-29a6c5913860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:13.702027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7c92b-4ee2-4a10-8339-29a6c5913860 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:13.702366 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7c92b-4ee2-4a10-8339-29a6c5913860 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:13.702669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7c92b-4ee2-4a10-8339-29a6c5913860 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:13.703296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7c92b-4ee2-4a10-8339-29a6c5913860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:13.705719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7c92b-4ee2-4a10-8339-29a6c5913860 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:13.792829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1237/2428] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:04:13 2022] POST /identity/v3/users => generated 283 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:04:13.796974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e8c1e-1056-405c-a2ea-af38762401fc admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:13.808689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e8c1e-1056-405c-a2ea-af38762401fc admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:13.813355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c7976-2850-4140-aa57-b67edb551de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:13.814124 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c7976-2850-4140-aa57-b67edb551de5 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:13.814474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c7976-2850-4140-aa57-b67edb551de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:13.814843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c7976-2850-4140-aa57-b67edb551de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:13.815664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c7976-2850-4140-aa57-b67edb551de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:13.817367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c7976-2850-4140-aa57-b67edb551de5 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:13.830409 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1192/2429] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:04:13.836138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7c92b-4ee2-4a10-8339-29a6c5913860 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:13.840065 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082836e-0b35-4d51-bc73-43b9aa489a42 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:13.840429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082836e-0b35-4d51-bc73-43b9aa489a42 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:13.840819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082836e-0b35-4d51-bc73-43b9aa489a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:13.849659 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7c92b-4ee2-4a10-8339-29a6c5913860 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:13.855578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a957cc1-b90c-482b-ab9d-dc31c3e18783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:13.859065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957cc1-b90c-482b-ab9d-dc31c3e18783 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:13.859065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957cc1-b90c-482b-ab9d-dc31c3e18783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:13.859065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957cc1-b90c-482b-ab9d-dc31c3e18783 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:13.859065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957cc1-b90c-482b-ab9d-dc31c3e18783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:13.859445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957cc1-b90c-482b-ab9d-dc31c3e18783 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:13.861153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:13.862308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6082836e-0b35-4d51-bc73-43b9aa489a42 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:13.863536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6082836e-0b35-4d51-bc73-43b9aa489a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:13.872523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1238/2430] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:04:13.885992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a957cc1-b90c-482b-ab9d-dc31c3e18783 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:13.896331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a957cc1-b90c-482b-ab9d-dc31c3e18783 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:13.902932 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5eeba-050a-4e0e-aef1-23225f37f1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:13.905046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5eeba-050a-4e0e-aef1-23225f37f1da admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:13.908103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5eeba-050a-4e0e-aef1-23225f37f1da admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:13.908103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5eeba-050a-4e0e-aef1-23225f37f1da admin admin] PATH_INFO: `/v3/projects/ce7a0226fc434949b56b9a21cd6d6c4d/users/be1ed37e13ad4495ae69f78d0a7768a6/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:13.923194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5eeba-050a-4e0e-aef1-23225f37f1da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7a0226fc434949b56b9a21cd6d6c4d, user_id=be1ed37e13ad4495ae69f78d0a7768a6, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:13.927712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5eeba-050a-4e0e-aef1-23225f37f1da admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:13.945738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1193/2431] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:04:13.968592 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082836e-0b35-4d51-bc73-43b9aa489a42 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:13.971740 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6082836e-0b35-4d51-bc73-43b9aa489a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:13.991164 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1239/2432] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:04:13 2022] PUT /identity/v3/projects/ce7a0226fc434949b56b9a21cd6d6c4d/users/be1ed37e13ad4495ae69f78d0a7768a6/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:04:14.007711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402fc2-cb4e-4dad-8a34-345605cd6ca4 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:14.008142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402fc2-cb4e-4dad-8a34-345605cd6ca4 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:14.009940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402fc2-cb4e-4dad-8a34-345605cd6ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:14.049617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d7402fc2-cb4e-4dad-8a34-345605cd6ca4 None None] MFA Rules not processed for user `be1ed37e13ad4495ae69f78d0a7768a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:14.050263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:14.051997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d7402fc2-cb4e-4dad-8a34-345605cd6ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:14.084603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082836e-0b35-4d51-bc73-43b9aa489a42 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:14.140053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1240/2433] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:04:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:04:14.152816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7402fc2-cb4e-4dad-8a34-345605cd6ca4 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:14.157597 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d7402fc2-cb4e-4dad-8a34-345605cd6ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:14.163106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cab974-0d34-418f-ac69-f425644c32d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:14.164485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cab974-0d34-418f-ac69-f425644c32d4 service swift] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:14.164485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cab974-0d34-418f-ac69-f425644c32d4 service swift] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:14.164485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cab974-0d34-418f-ac69-f425644c32d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:14.233298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7402fc2-cb4e-4dad-8a34-345605cd6ca4 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:14.239264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07457fb-bce3-4c5f-aead-49c5d7e50d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:14.240221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07457fb-bce3-4c5f-aead-49c5d7e50d97 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:14.240508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07457fb-bce3-4c5f-aead-49c5d7e50d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:14.240832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07457fb-bce3-4c5f-aead-49c5d7e50d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:14.250258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cab974-0d34-418f-ac69-f425644c32d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:14.251631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cab974-0d34-418f-ac69-f425644c32d4 service swift] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:14.255905 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07457fb-bce3-4c5f-aead-49c5d7e50d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:14.257622 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07457fb-bce3-4c5f-aead-49c5d7e50d97 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:14.261193 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1194/2434] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:04:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 24 20:04:14.268857 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1241/2435] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:04:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:04:24.263732 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=70,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:24.264587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=58 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:29.388715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db7e7f-7174-4f31-b5dc-8d04274ea2b2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:29.389348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db7e7f-7174-4f31-b5dc-8d04274ea2b2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:29.389348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db7e7f-7174-4f31-b5dc-8d04274ea2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:29.407108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f8db7e7f-7174-4f31-b5dc-8d04274ea2b2 None None] MFA Rules not processed for user `7ed43701d36b4b479a2a3a978d242625`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:29.408348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f8db7e7f-7174-4f31-b5dc-8d04274ea2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:29.480864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1195/2436] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:04:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:32.497092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6af395-6d25-48db-8f35-3ffd0ca11092 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:32.497585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6af395-6d25-48db-8f35-3ffd0ca11092 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:32.498101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6af395-6d25-48db-8f35-3ffd0ca11092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:32.525353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-9e6af395-6d25-48db-8f35-3ffd0ca11092 None None] MFA Rules not processed for user `822e56f086f0481ca4c6970adf85e396`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:32.527362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9e6af395-6d25-48db-8f35-3ffd0ca11092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:32.586703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1242/2437] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:04:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:38.329069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6547b6-2bb4-4ed2-a4cb-73ccfb781f3c None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:38.331201 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6547b6-2bb4-4ed2-a4cb-73ccfb781f3c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:38.332012 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6547b6-2bb4-4ed2-a4cb-73ccfb781f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:38.366148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-1a6547b6-2bb4-4ed2-a4cb-73ccfb781f3c None None] MFA Rules not processed for user `46a641c6357148f984bd21afe6cfb242`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:38.368544 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1a6547b6-2bb4-4ed2-a4cb-73ccfb781f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:38.571433 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1196/2438] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:04:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:39.079520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6af395-6d25-48db-8f35-3ffd0ca11092 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:39.081728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9e6af395-6d25-48db-8f35-3ffd0ca11092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:39.200249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6af395-6d25-48db-8f35-3ffd0ca11092 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:39.205753 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4a28c-51d0-4254-9c59-6038d1b7fa28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:39.207302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4a28c-51d0-4254-9c59-6038d1b7fa28 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:39.207821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4a28c-51d0-4254-9c59-6038d1b7fa28 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:39.208544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4a28c-51d0-4254-9c59-6038d1b7fa28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:39.211290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8ef4a28c-51d0-4254-9c59-6038d1b7fa28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:39.364123 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4a28c-51d0-4254-9c59-6038d1b7fa28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:39.366902 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4a28c-51d0-4254-9c59-6038d1b7fa28 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:39.380243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1243/2439] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:04:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:04:42.299481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ad15a-2765-4f30-aa26-691c21ef21f5 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:42.300438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ad15a-2765-4f30-aa26-691c21ef21f5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:42.300973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ad15a-2765-4f30-aa26-691c21ef21f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:42.335100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-f56ad15a-2765-4f30-aa26-691c21ef21f5 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:04:42.337758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f56ad15a-2765-4f30-aa26-691c21ef21f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:04:42.377369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1197/2440] 10.222.0.17 () {58 vars in 967 bytes} [Sun Jul 24 20:04:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:04:42.405662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4a28c-51d0-4254-9c59-6038d1b7fa28 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:42.437821 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4a28c-51d0-4254-9c59-6038d1b7fa28 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:42.447170 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fe17a-b01a-41ba-ae36-622d8d9198af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:42.448830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fe17a-b01a-41ba-ae36-622d8d9198af service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:42.449521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fe17a-b01a-41ba-ae36-622d8d9198af service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:42.450122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fe17a-b01a-41ba-ae36-622d8d9198af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:42.467135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fe17a-b01a-41ba-ae36-622d8d9198af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:42.471110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fe17a-b01a-41ba-ae36-622d8d9198af service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:04:42.485660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1244/2441] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:04:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 24 20:04:52.326195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=53 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:52.480900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=81 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:04:59.657938 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ad15a-2765-4f30-aa26-691c21ef21f5 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:04:59.747653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ad15a-2765-4f30-aa26-691c21ef21f5 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:04:59.755681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:04:59.757312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:04:59.757839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:04:59.758362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] PATH_INFO: `/v3/users/be1ed37e13ad4495ae69f78d0a7768a6` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:04:59.761806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1ed37e13ad4495ae69f78d0a7768a6)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:04:59.764407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:00.054481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1ed37e13ad4495ae69f78d0a7768a6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:00.065049 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1ed37e13ad4495ae69f78d0a7768a6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:00.065963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1ed37e13ad4495ae69f78d0a7768a6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:00.187491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1ed37e13ad4495ae69f78d0a7768a6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:00.203095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1ed37e13ad4495ae69f78d0a7768a6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:00.212103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1198/2442] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:04:59 2022] DELETE /identity/v3/users/be1ed37e13ad4495ae69f78d0a7768a6 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:05:00.228875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f07c94-82f8-451f-823b-8c6fe28f0d7c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:00.228875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f07c94-82f8-451f-823b-8c6fe28f0d7c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:00.228875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f07c94-82f8-451f-823b-8c6fe28f0d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:00.241651 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c6f07c94-82f8-451f-823b-8c6fe28f0d7c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:05:00.243078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c6f07c94-82f8-451f-823b-8c6fe28f0d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:00.312385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1245/2443] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:05:00.322797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:00.324481 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:00.397769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b39bfd-0f36-4dfd-badf-a53a1c9a4a96 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:00.402183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231477b8-f4a7-4ed8-a62a-db389633e283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:00.402911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231477b8-f4a7-4ed8-a62a-db389633e283 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:00.403194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231477b8-f4a7-4ed8-a62a-db389633e283 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:00.403500 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231477b8-f4a7-4ed8-a62a-db389633e283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:00.415010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231477b8-f4a7-4ed8-a62a-db389633e283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:00.417345 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231477b8-f4a7-4ed8-a62a-db389633e283 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:00.433416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1199/2444] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:05:01.175012 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f07c94-82f8-451f-823b-8c6fe28f0d7c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:01.178522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c6f07c94-82f8-451f-823b-8c6fe28f0d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:01.210484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f07c94-82f8-451f-823b-8c6fe28f0d7c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:01.234520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:01.236061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:01.236888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:01.237487 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] PATH_INFO: `/v3/projects/ce7a0226fc434949b56b9a21cd6d6c4d` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:01.239791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce7a0226fc434949b56b9a21cd6d6c4d)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:01.245219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:01.464459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce7a0226fc434949b56b9a21cd6d6c4d', 'request_id': 'req-289d2719-1326-4609-af23-b8aeb1b9e4f2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:01.478022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce7a0226fc434949b56b9a21cd6d6c4d', 'request_id': 'req-289d2719-1326-4609-af23-b8aeb1b9e4f2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:01.488598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce7a0226fc434949b56b9a21cd6d6c4d', 'request_id': 'req-289d2719-1326-4609-af23-b8aeb1b9e4f2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:01.590973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] The token cache is being invalidate because project ce7a0226fc434949b56b9a21cd6d6c4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:05:01.591488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:01.592611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-289d2719-1326-4609-af23-b8aeb1b9e4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:01.595476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1246/2445] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:05:01 2022] DELETE /identity/v3/projects/ce7a0226fc434949b56b9a21cd6d6c4d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:05:04.087527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231477b8-f4a7-4ed8-a62a-db389633e283 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:04.098152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231477b8-f4a7-4ed8-a62a-db389633e283 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:04.121442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:05:04.163833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:04.164837 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:04.165077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:04.165401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:04.168574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:04.289328 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:04.290914 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:04.305614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1200/2446] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:05:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:05:11.584635 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:05:14.298153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=40 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:05:14.401901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee46932-85a5-4446-8fd8-f13b8248bf9a None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:14.402657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee46932-85a5-4446-8fd8-f13b8248bf9a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:14.402988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee46932-85a5-4446-8fd8-f13b8248bf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:14.429950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-3ee46932-85a5-4446-8fd8-f13b8248bf9a None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:05:14.431558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3ee46932-85a5-4446-8fd8-f13b8248bf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:14.521592 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1247/2447] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:05:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:05:14.552859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:14.556125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:14.696704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed524-7ea8-460f-9368-26f23cc414be service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:14.705901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f59b85-fbef-4944-9693-e08ebe1ac54a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:14.707087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f59b85-fbef-4944-9693-e08ebe1ac54a service swift] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:14.708164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f59b85-fbef-4944-9693-e08ebe1ac54a service swift] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:14.711393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f59b85-fbef-4944-9693-e08ebe1ac54a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:14.733023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f59b85-fbef-4944-9693-e08ebe1ac54a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:14.736177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f59b85-fbef-4944-9693-e08ebe1ac54a service swift] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:14.752744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1201/2448] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:05:17.753903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb098f-66db-467a-b800-378b17b323b1 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:17.757510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb098f-66db-467a-b800-378b17b323b1 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:17.758915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb098f-66db-467a-b800-378b17b323b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:17.797076 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-3ebb098f-66db-467a-b800-378b17b323b1 None None] MFA Rules not processed for user `5221431e0552435d9ff82f8ce75b624e`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:05:17.799309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3ebb098f-66db-467a-b800-378b17b323b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:17.914818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1248/2449] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:05:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:05:24.746335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=32 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:05:27.900929 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=50 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:05:35.113709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a99706-501c-4761-bd00-48bc12d22bea None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:35.114594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a99706-501c-4761-bd00-48bc12d22bea None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:35.114771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a99706-501c-4761-bd00-48bc12d22bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:35.137894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-52a99706-501c-4761-bd00-48bc12d22bea None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:05:35.139430 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-52a99706-501c-4761-bd00-48bc12d22bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:35.149335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1202/2450] 10.222.0.17 () {58 vars in 967 bytes} [Sun Jul 24 20:05:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:05:35.162819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb098f-66db-467a-b800-378b17b323b1 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:35.177878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb098f-66db-467a-b800-378b17b323b1 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:35.184598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b91545-45c8-4c12-9898-883c946fef0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:35.185318 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b91545-45c8-4c12-9898-883c946fef0f service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:35.185642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b91545-45c8-4c12-9898-883c946fef0f service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:35.186040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b91545-45c8-4c12-9898-883c946fef0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:35.196199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b91545-45c8-4c12-9898-883c946fef0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:35.198106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b91545-45c8-4c12-9898-883c946fef0f service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:35.208294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1249/2451] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:05:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:05:38.356855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a99706-501c-4761-bd00-48bc12d22bea None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:38.358506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-52a99706-501c-4761-bd00-48bc12d22bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:38.437527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a99706-501c-4761-bd00-48bc12d22bea None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:38.442378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:38.443277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:38.446858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:38.446858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] PATH_INFO: `/v3/users/46a641c6357148f984bd21afe6cfb242` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:38.446858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a641c6357148f984bd21afe6cfb242)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:38.447242 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:38.652138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a641c6357148f984bd21afe6cfb242'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:38.660195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a641c6357148f984bd21afe6cfb242'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:38.661005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a641c6357148f984bd21afe6cfb242'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:38.759464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a641c6357148f984bd21afe6cfb242'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:38.775997 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0ae893e2-6b67-4afd-99f7-0e0c612efecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a641c6357148f984bd21afe6cfb242'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:38.783991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1203/2452] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:05:38 2022] DELETE /identity/v3/users/46a641c6357148f984bd21afe6cfb242 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:05:39.060068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b91545-45c8-4c12-9898-883c946fef0f service swift] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:39.077051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b91545-45c8-4c12-9898-883c946fef0f service swift] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:39.175560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:39.176381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:39.176698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:39.176982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] PATH_INFO: `/v3/projects/22e598739ae64ebf9e523e4e2e6a2867` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:39.178293 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e598739ae64ebf9e523e4e2e6a2867)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:39.180580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:39.349928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e598739ae64ebf9e523e4e2e6a2867', 'request_id': 'req-7a78a241-04bf-44cd-b534-d21fd9b455fd'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:39.355533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e598739ae64ebf9e523e4e2e6a2867', 'request_id': 'req-7a78a241-04bf-44cd-b534-d21fd9b455fd'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:39.362448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e598739ae64ebf9e523e4e2e6a2867', 'request_id': 'req-7a78a241-04bf-44cd-b534-d21fd9b455fd'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:39.440334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] The token cache is being invalidate because project 22e598739ae64ebf9e523e4e2e6a2867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:05:39.440616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:39.441321 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7a78a241-04bf-44cd-b534-d21fd9b455fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:05:39.442920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1250/2453] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:05:39 2022] DELETE /identity/v3/projects/22e598739ae64ebf9e523e4e2e6a2867 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:05:39.450010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6d4c7-0a0e-4777-afe0-6d1e16b752b0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:39.450657 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6d4c7-0a0e-4777-afe0-6d1e16b752b0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:39.451098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6d4c7-0a0e-4777-afe0-6d1e16b752b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:39.476593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-dcd6d4c7-0a0e-4777-afe0-6d1e16b752b0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:05:39.477892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-dcd6d4c7-0a0e-4777-afe0-6d1e16b752b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:39.548287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1204/2454] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:05:39.558696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7c8ca-b7c0-4c52-ac95-019c3a88972a None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:39.559061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7c8ca-b7c0-4c52-ac95-019c3a88972a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:39.559407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7c8ca-b7c0-4c52-ac95-019c3a88972a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:39.589334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-aac7c8ca-b7c0-4c52-ac95-019c3a88972a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:05:39.591461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-aac7c8ca-b7c0-4c52-ac95-019c3a88972a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:39.605886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1251/2455] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:05:39.610393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6d4c7-0a0e-4777-afe0-6d1e16b752b0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:39.622162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6d4c7-0a0e-4777-afe0-6d1e16b752b0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:39.627598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572f578-0451-49e3-9b40-bedf46f827fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:39.628100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572f578-0451-49e3-9b40-bedf46f827fe admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:39.628270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572f578-0451-49e3-9b40-bedf46f827fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:39.628554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572f578-0451-49e3-9b40-bedf46f827fe admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:39.629332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572f578-0451-49e3-9b40-bedf46f827fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:05:39.629654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572f578-0451-49e3-9b40-bedf46f827fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:39.631283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572f578-0451-49e3-9b40-bedf46f827fe admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:39.641163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:05:39.645125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1205/2456] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:05:39.649569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7c8ca-b7c0-4c52-ac95-019c3a88972a None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:39.659274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7c8ca-b7c0-4c52-ac95-019c3a88972a None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:39.666023 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ac31f-2bb2-45f4-95c9-e2d14dbb52b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:39.666716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ac31f-2bb2-45f4-95c9-e2d14dbb52b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:39.667607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ac31f-2bb2-45f4-95c9-e2d14dbb52b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:39.667879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ac31f-2bb2-45f4-95c9-e2d14dbb52b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:39.668616 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ac31f-2bb2-45f4-95c9-e2d14dbb52b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:39.670714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ac31f-2bb2-45f4-95c9-e2d14dbb52b8 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:39.813726 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1252/2457] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:05:39 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:05:39.819840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572f578-0451-49e3-9b40-bedf46f827fe admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:39.832615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572f578-0451-49e3-9b40-bedf46f827fe admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:39.897774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e903493-faef-442d-a1ae-a018ae95be42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:39.898474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e903493-faef-442d-a1ae-a018ae95be42 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:39.898762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e903493-faef-442d-a1ae-a018ae95be42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:39.899042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e903493-faef-442d-a1ae-a018ae95be42 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:39.899752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e903493-faef-442d-a1ae-a018ae95be42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:39.901909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e903493-faef-442d-a1ae-a018ae95be42 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:40.005524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1206/2458] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:05:39 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:05:40.009832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ac31f-2bb2-45f4-95c9-e2d14dbb52b8 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:40.023854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ac31f-2bb2-45f4-95c9-e2d14dbb52b8 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:40.031121 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e00300-2695-4465-b4f1-a557edfc7da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:40.032393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e00300-2695-4465-b4f1-a557edfc7da1 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:40.032943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e00300-2695-4465-b4f1-a557edfc7da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:40.033461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e00300-2695-4465-b4f1-a557edfc7da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:40.034833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e00300-2695-4465-b4f1-a557edfc7da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:40.038050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e00300-2695-4465-b4f1-a557edfc7da1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:40.051974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1253/2459] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:05:40.057139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e903493-faef-442d-a1ae-a018ae95be42 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:40.068890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e903493-faef-442d-a1ae-a018ae95be42 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:40.075436 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27733a87-344f-46d1-acca-d751eba42f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:40.076761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27733a87-344f-46d1-acca-d751eba42f08 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:40.076761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27733a87-344f-46d1-acca-d751eba42f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:40.077011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27733a87-344f-46d1-acca-d751eba42f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:40.077537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27733a87-344f-46d1-acca-d751eba42f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:40.079186 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27733a87-344f-46d1-acca-d751eba42f08 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:40.090770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1207/2460] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:05:40.095417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e00300-2695-4465-b4f1-a557edfc7da1 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:40.106461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e00300-2695-4465-b4f1-a557edfc7da1 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:40.112209 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d5270-3467-43ac-b76f-2604e6448f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:40.113167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d5270-3467-43ac-b76f-2604e6448f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:40.113455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d5270-3467-43ac-b76f-2604e6448f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:40.113741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d5270-3467-43ac-b76f-2604e6448f28 admin admin] PATH_INFO: `/v3/projects/d4f1f24b3a384f1db3a6e782253814ec/users/427fa281fc374bd2b9cd7b7fe59297a3/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:40.131927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d5270-3467-43ac-b76f-2604e6448f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f1f24b3a384f1db3a6e782253814ec, user_id=427fa281fc374bd2b9cd7b7fe59297a3, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:40.134639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d5270-3467-43ac-b76f-2604e6448f28 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:40.224049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1254/2461] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:05:40 2022] PUT /identity/v3/projects/d4f1f24b3a384f1db3a6e782253814ec/users/427fa281fc374bd2b9cd7b7fe59297a3/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:05:40.231614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1cf8b-5f03-44fb-8729-99e27473f2f5 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:40.232046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1cf8b-5f03-44fb-8729-99e27473f2f5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:40.233373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1cf8b-5f03-44fb-8729-99e27473f2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:40.254231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-82d1cf8b-5f03-44fb-8729-99e27473f2f5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:05:40.260401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-82d1cf8b-5f03-44fb-8729-99e27473f2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:40.324071 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1208/2462] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:05:40.334733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d5270-3467-43ac-b76f-2604e6448f28 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:40.336727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9c4d5270-3467-43ac-b76f-2604e6448f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:40.353906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:05:40.407327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d5270-3467-43ac-b76f-2604e6448f28 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:40.411262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10901-1ee6-4b88-9daa-ceee8b261bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:40.411935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10901-1ee6-4b88-9daa-ceee8b261bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:40.412221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10901-1ee6-4b88-9daa-ceee8b261bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:40.412501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10901-1ee6-4b88-9daa-ceee8b261bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:40.427479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed10901-1ee6-4b88-9daa-ceee8b261bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:40.427479 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed10901-1ee6-4b88-9daa-ceee8b261bf9 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:40.437459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1255/2463] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:05:49.882707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf03f48-56be-4d96-934c-32af8a0989e0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:49.883579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf03f48-56be-4d96-934c-32af8a0989e0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:49.883579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf03f48-56be-4d96-934c-32af8a0989e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:49.923047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-7bf03f48-56be-4d96-934c-32af8a0989e0 None None] MFA Rules not processed for user `427fa281fc374bd2b9cd7b7fe59297a3`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:05:49.934966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7bf03f48-56be-4d96-934c-32af8a0989e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:50.068711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1209/2464] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:05:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:05:50.074237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10901-1ee6-4b88-9daa-ceee8b261bf9 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:50.084990 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10901-1ee6-4b88-9daa-ceee8b261bf9 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:50.090496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3363015-4dc8-4ea7-b698-6dab82e80106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:50.091174 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3363015-4dc8-4ea7-b698-6dab82e80106 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:50.091629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3363015-4dc8-4ea7-b698-6dab82e80106 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:50.091913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3363015-4dc8-4ea7-b698-6dab82e80106 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:50.092632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3363015-4dc8-4ea7-b698-6dab82e80106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:50.097150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3363015-4dc8-4ea7-b698-6dab82e80106 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:50.232895 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1256/2465] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:05:50 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:05:50.237132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf03f48-56be-4d96-934c-32af8a0989e0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:50.252942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf03f48-56be-4d96-934c-32af8a0989e0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:50.262382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:05:50.319472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c9df2-8fc7-4d7d-9f6a-bd7e3b96c901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:50.320161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9df2-8fc7-4d7d-9f6a-bd7e3b96c901 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:50.321121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9df2-8fc7-4d7d-9f6a-bd7e3b96c901 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:50.321121 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9df2-8fc7-4d7d-9f6a-bd7e3b96c901 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:50.321487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c9df2-8fc7-4d7d-9f6a-bd7e3b96c901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:50.324035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c9df2-8fc7-4d7d-9f6a-bd7e3b96c901 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:50.531030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1210/2466] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:05:50 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:05:50.535111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3363015-4dc8-4ea7-b698-6dab82e80106 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:50.546612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3363015-4dc8-4ea7-b698-6dab82e80106 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:50.552157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3d85b-3f68-45ab-9ec5-dfe577971002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:50.552984 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3d85b-3f68-45ab-9ec5-dfe577971002 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:50.553280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3d85b-3f68-45ab-9ec5-dfe577971002 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:50.553593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3d85b-3f68-45ab-9ec5-dfe577971002 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:50.554317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3d85b-3f68-45ab-9ec5-dfe577971002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:50.556767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3d85b-3f68-45ab-9ec5-dfe577971002 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:50.570550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1257/2467] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:05:50.578711 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c9df2-8fc7-4d7d-9f6a-bd7e3b96c901 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:50.601634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c9df2-8fc7-4d7d-9f6a-bd7e3b96c901 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:50.610805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927843b0-53e3-4ffe-989a-93a657dd4c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:50.612509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927843b0-53e3-4ffe-989a-93a657dd4c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:50.633708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927843b0-53e3-4ffe-989a-93a657dd4c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:50.633708 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927843b0-53e3-4ffe-989a-93a657dd4c25 admin admin] PATH_INFO: `/v3/projects/fd7606f6fdc54b33b7490691669a08a2/users/bd25fe6aed024fc6b75f1bf449ce7321/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:50.655471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927843b0-53e3-4ffe-989a-93a657dd4c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7606f6fdc54b33b7490691669a08a2, user_id=bd25fe6aed024fc6b75f1bf449ce7321, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:50.660356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927843b0-53e3-4ffe-989a-93a657dd4c25 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:50.755008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1211/2468] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:05:50 2022] PUT /identity/v3/projects/fd7606f6fdc54b33b7490691669a08a2/users/bd25fe6aed024fc6b75f1bf449ce7321/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:05:50.760352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3d85b-3f68-45ab-9ec5-dfe577971002 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:50.773394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3d85b-3f68-45ab-9ec5-dfe577971002 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:50.875629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de177069-1544-4c2d-9fcb-213663844185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:50.876759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de177069-1544-4c2d-9fcb-213663844185 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:50.877713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de177069-1544-4c2d-9fcb-213663844185 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:50.879035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de177069-1544-4c2d-9fcb-213663844185 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:50.880955 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de177069-1544-4c2d-9fcb-213663844185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:50.884900 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de177069-1544-4c2d-9fcb-213663844185 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:50.914924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1258/2469] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:05:50.930893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927843b0-53e3-4ffe-989a-93a657dd4c25 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:05:50.937975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927843b0-53e3-4ffe-989a-93a657dd4c25 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:05:50.946295 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d796cf8-e3b3-443b-a538-3198b29f1caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:05:50.948326 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d796cf8-e3b3-443b-a538-3198b29f1caf admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:50.949487 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d796cf8-e3b3-443b-a538-3198b29f1caf admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:50.950340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d796cf8-e3b3-443b-a538-3198b29f1caf admin admin] PATH_INFO: `/v3/projects/fd7606f6fdc54b33b7490691669a08a2/users/bd25fe6aed024fc6b75f1bf449ce7321/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:50.953924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d796cf8-e3b3-443b-a538-3198b29f1caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7606f6fdc54b33b7490691669a08a2, user_id=bd25fe6aed024fc6b75f1bf449ce7321, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:05:50.959040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d796cf8-e3b3-443b-a538-3198b29f1caf admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:05:50.986598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1212/2470] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:05:50 2022] PUT /identity/v3/projects/fd7606f6fdc54b33b7490691669a08a2/users/bd25fe6aed024fc6b75f1bf449ce7321/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:05:52.407565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3e3b9-60ad-4b3c-ada7-f03a138f0a95 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:05:52.408152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3e3b9-60ad-4b3c-ada7-f03a138f0a95 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:05:52.408152 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3e3b9-60ad-4b3c-ada7-f03a138f0a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:05:52.428198 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a3e3e3b9-60ad-4b3c-ada7-f03a138f0a95 None None] MFA Rules not processed for user `55ed193c93c446a8b373f770512e5160`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:05:52.429748 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a3e3e3b9-60ad-4b3c-ada7-f03a138f0a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:05:52.516197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1259/2471] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:05:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:00.001768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f76c98-3ba5-4516-aa42-2b7bff52d92a None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:00.002845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f76c98-3ba5-4516-aa42-2b7bff52d92a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:00.003760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f76c98-3ba5-4516-aa42-2b7bff52d92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:00.041314 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-83f76c98-3ba5-4516-aa42-2b7bff52d92a None None] MFA Rules not processed for user `bd25fe6aed024fc6b75f1bf449ce7321`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:00.042949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-83f76c98-3ba5-4516-aa42-2b7bff52d92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:00.147504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1213/2472] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:05:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:00.162968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3e3b9-60ad-4b3c-ada7-f03a138f0a95 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:00.165047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a3e3e3b9-60ad-4b3c-ada7-f03a138f0a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:00.203505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:00.282758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3e3b9-60ad-4b3c-ada7-f03a138f0a95 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:00.290188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47001e-8c2b-4c3f-8e69-fa4e19421c46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:00.291209 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47001e-8c2b-4c3f-8e69-fa4e19421c46 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:00.291496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47001e-8c2b-4c3f-8e69-fa4e19421c46 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:00.291791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47001e-8c2b-4c3f-8e69-fa4e19421c46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:00.356368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47001e-8c2b-4c3f-8e69-fa4e19421c46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:00.357960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47001e-8c2b-4c3f-8e69-fa4e19421c46 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:00.372003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1260/2473] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:04.563969 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6d522-7c58-4438-9836-dbf0181a2e14 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:04.564845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6d522-7c58-4438-9836-dbf0181a2e14 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:04.564845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6d522-7c58-4438-9836-dbf0181a2e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:04.584772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-83c6d522-7c58-4438-9836-dbf0181a2e14 None None] MFA Rules not processed for user `427fa281fc374bd2b9cd7b7fe59297a3`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:04.586244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-83c6d522-7c58-4438-9836-dbf0181a2e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:04.594469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1214/2474] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:06:04.961099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddafe7-78a4-4ffc-a75c-b3eb977355af None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:04.961669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddafe7-78a4-4ffc-a75c-b3eb977355af None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:04.961669 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddafe7-78a4-4ffc-a75c-b3eb977355af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:04.986441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-10ddafe7-78a4-4ffc-a75c-b3eb977355af None None] MFA Rules not processed for user `822e56f086f0481ca4c6970adf85e396`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:04.987668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-10ddafe7-78a4-4ffc-a75c-b3eb977355af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:05.064151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1261/2475] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:05.975564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6d522-7c58-4438-9836-dbf0181a2e14 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:06.105595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:06.143731 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6d522-7c58-4438-9836-dbf0181a2e14 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:06.147582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a0d37-7ac9-4a86-88e2-ad70acc79d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:06.148286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a0d37-7ac9-4a86-88e2-ad70acc79d61 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:06.148573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a0d37-7ac9-4a86-88e2-ad70acc79d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:06.148885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a0d37-7ac9-4a86-88e2-ad70acc79d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:06.151075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2a4a0d37-7ac9-4a86-88e2-ad70acc79d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:06.167024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a0d37-7ac9-4a86-88e2-ad70acc79d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:06.171160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a0d37-7ac9-4a86-88e2-ad70acc79d61 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:06.188198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1215/2476] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:15.053494 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=70 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:17.009047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddafe7-78a4-4ffc-a75c-b3eb977355af None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:17.020524 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddafe7-78a4-4ffc-a75c-b3eb977355af None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:17.024452 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddf7bd-eff4-4b09-83ab-e5a0d228368a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:17.025149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddf7bd-eff4-4b09-83ab-e5a0d228368a service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:17.025436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddf7bd-eff4-4b09-83ab-e5a0d228368a service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:17.025719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddf7bd-eff4-4b09-83ab-e5a0d228368a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:17.028260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-43ddf7bd-eff4-4b09-83ab-e5a0d228368a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:17.102347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=8 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:17.106983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddf7bd-eff4-4b09-83ab-e5a0d228368a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:17.108452 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddf7bd-eff4-4b09-83ab-e5a0d228368a service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:17.119248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1262/2477] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:18.408848 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a0d37-7ac9-4a86-88e2-ad70acc79d61 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:18.418795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a0d37-7ac9-4a86-88e2-ad70acc79d61 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:18.422950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:18.426271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:18.426271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:18.426271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:18.426753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:18.442324 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:18.444095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:18.460760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1216/2478] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:22.043442 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddf7bd-eff4-4b09-83ab-e5a0d228368a service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:22.053017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddf7bd-eff4-4b09-83ab-e5a0d228368a service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:22.057193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc7e6e-8406-49f7-94ad-204ba65a6f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:22.057916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc7e6e-8406-49f7-94ad-204ba65a6f73 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:22.058376 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc7e6e-8406-49f7-94ad-204ba65a6f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:22.058544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc7e6e-8406-49f7-94ad-204ba65a6f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:22.060266 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-eddc7e6e-8406-49f7-94ad-204ba65a6f73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:22.070987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc7e6e-8406-49f7-94ad-204ba65a6f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:22.072714 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc7e6e-8406-49f7-94ad-204ba65a6f73 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:22.081997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1263/2479] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:23.090677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:23.092457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:23.196634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a433e5-2f44-4312-b0dc-bd14ccc746e2 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:23.203839 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9884bd-ab31-4149-8fa6-3df479cad0bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:23.204530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9884bd-ab31-4149-8fa6-3df479cad0bf service placement] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:23.204863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9884bd-ab31-4149-8fa6-3df479cad0bf service placement] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:23.205145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9884bd-ab31-4149-8fa6-3df479cad0bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:23.207234 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-af9884bd-ab31-4149-8fa6-3df479cad0bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:23.220303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9884bd-ab31-4149-8fa6-3df479cad0bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:23.221982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9884bd-ab31-4149-8fa6-3df479cad0bf service placement] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:23.233163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1217/2480] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:32.078749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=34 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:32.902138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc7e6e-8406-49f7-94ad-204ba65a6f73 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:32.914639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc7e6e-8406-49f7-94ad-204ba65a6f73 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:32.920446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:32.925128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:32.925128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:32.925128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:32.925128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:33.035969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:33.044605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:33.066461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1264/2481] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:06:33.228578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=38 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:34.090079 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9884bd-ab31-4149-8fa6-3df479cad0bf service placement] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:34.102755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9884bd-ab31-4149-8fa6-3df479cad0bf service placement] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:34.109407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf82db-c398-480f-80b1-8fff6a4050ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:34.110276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf82db-c398-480f-80b1-8fff6a4050ec service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:34.110555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf82db-c398-480f-80b1-8fff6a4050ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:34.111029 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf82db-c398-480f-80b1-8fff6a4050ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:34.113109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fabf82db-c398-480f-80b1-8fff6a4050ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:34.126044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf82db-c398-480f-80b1-8fff6a4050ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:34.128393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf82db-c398-480f-80b1-8fff6a4050ec service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:34.138176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1218/2482] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:39.521089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:39.522797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:39.537589 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c30257-a653-40c3-a2e8-2875deb75db3 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:39.542710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:39.543608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:39.543896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:39.544180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] PATH_INFO: `/v3/users/5221431e0552435d9ff82f8ce75b624e` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:39.546114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5221431e0552435d9ff82f8ce75b624e)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:39.548166 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:39.757031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5221431e0552435d9ff82f8ce75b624e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:39.778996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5221431e0552435d9ff82f8ce75b624e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:39.786757 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5221431e0552435d9ff82f8ce75b624e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:39.787559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5221431e0552435d9ff82f8ce75b624e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:39.855629 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-df38eaaf-69eb-4f37-a1de-5e842b33f812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5221431e0552435d9ff82f8ce75b624e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:39.867002 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1265/2483] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:06:39 2022] DELETE /identity/v3/users/5221431e0552435d9ff82f8ce75b624e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:40.168756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf82db-c398-480f-80b1-8fff6a4050ec service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:40.183628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf82db-c398-480f-80b1-8fff6a4050ec service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:40.245885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:40.249974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:40.249974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:40.249974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] PATH_INFO: `/v3/projects/15c454baefdc474682f88c603f6108a8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:40.250754 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c454baefdc474682f88c603f6108a8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:40.254640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:40.475338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c454baefdc474682f88c603f6108a8', 'request_id': 'req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:40.482980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c454baefdc474682f88c603f6108a8', 'request_id': 'req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:40.600039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c454baefdc474682f88c603f6108a8', 'request_id': 'req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:40.614616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] The token cache is being invalidate because project 15c454baefdc474682f88c603f6108a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:06:40.618625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:40.618625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d83ee25a-550d-4597-a0cd-ac1d02d87f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:40.618625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1219/2484] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:06:40 2022] DELETE /identity/v3/projects/15c454baefdc474682f88c603f6108a8 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:40.623621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c2542-2123-4fae-8e46-de5fae004aaa None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:40.624206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c2542-2123-4fae-8e46-de5fae004aaa None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:40.624564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c2542-2123-4fae-8e46-de5fae004aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:40.645430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1f2c2542-2123-4fae-8e46-de5fae004aaa None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:40.649057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1f2c2542-2123-4fae-8e46-de5fae004aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:40.724799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1266/2485] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:40.732078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75b79-514e-4026-ac13-2eab47ea4e5a None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:40.732380 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75b79-514e-4026-ac13-2eab47ea4e5a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:40.732694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75b79-514e-4026-ac13-2eab47ea4e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:40.753091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-48e75b79-514e-4026-ac13-2eab47ea4e5a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:40.754344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-48e75b79-514e-4026-ac13-2eab47ea4e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:40.765241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1220/2486] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:40.769464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c2542-2123-4fae-8e46-de5fae004aaa None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:40.786090 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c2542-2123-4fae-8e46-de5fae004aaa None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:40.795700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281aedd-08e2-4a32-9aec-4acba1debc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:40.796608 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281aedd-08e2-4a32-9aec-4acba1debc25 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:40.797084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281aedd-08e2-4a32-9aec-4acba1debc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:40.797430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281aedd-08e2-4a32-9aec-4acba1debc25 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:40.798461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281aedd-08e2-4a32-9aec-4acba1debc25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:06:40.799062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281aedd-08e2-4a32-9aec-4acba1debc25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:40.801082 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281aedd-08e2-4a32-9aec-4acba1debc25 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:40.804331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:40.818248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1267/2487] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:06:40.823968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e75b79-514e-4026-ac13-2eab47ea4e5a None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:40.834562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e75b79-514e-4026-ac13-2eab47ea4e5a None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:40.840078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475fbca1-ca4d-48d2-897f-5fc2ddae5fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:40.840846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fbca1-ca4d-48d2-897f-5fc2ddae5fab admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:40.841211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fbca1-ca4d-48d2-897f-5fc2ddae5fab admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:40.841416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fbca1-ca4d-48d2-897f-5fc2ddae5fab admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:40.842136 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475fbca1-ca4d-48d2-897f-5fc2ddae5fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:40.844080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475fbca1-ca4d-48d2-897f-5fc2ddae5fab admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:40.960589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1221/2488] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:06:40 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:06:40.964919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281aedd-08e2-4a32-9aec-4acba1debc25 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:40.976142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281aedd-08e2-4a32-9aec-4acba1debc25 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:41.041853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec043b3f-160f-4c85-9d04-ac69a2629b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:41.043231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec043b3f-160f-4c85-9d04-ac69a2629b42 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:41.043516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec043b3f-160f-4c85-9d04-ac69a2629b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:41.043793 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec043b3f-160f-4c85-9d04-ac69a2629b42 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:41.044502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec043b3f-160f-4c85-9d04-ac69a2629b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:41.046559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec043b3f-160f-4c85-9d04-ac69a2629b42 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:41.136322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1268/2489] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:06:40 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:06:41.140747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475fbca1-ca4d-48d2-897f-5fc2ddae5fab admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:41.153179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475fbca1-ca4d-48d2-897f-5fc2ddae5fab admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:41.158151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc97b7c-8da6-4bf0-b743-de08004191b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:41.158827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc97b7c-8da6-4bf0-b743-de08004191b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:41.159111 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc97b7c-8da6-4bf0-b743-de08004191b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:41.159387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc97b7c-8da6-4bf0-b743-de08004191b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:41.160118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc97b7c-8da6-4bf0-b743-de08004191b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:41.161753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc97b7c-8da6-4bf0-b743-de08004191b9 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:41.174536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1222/2490] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:06:41.178649 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec043b3f-160f-4c85-9d04-ac69a2629b42 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:41.190742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec043b3f-160f-4c85-9d04-ac69a2629b42 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:41.195962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7164b65-b2b3-4e30-97f1-448cdd8d8d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:41.196660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7164b65-b2b3-4e30-97f1-448cdd8d8d53 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:41.196959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7164b65-b2b3-4e30-97f1-448cdd8d8d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:41.197237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7164b65-b2b3-4e30-97f1-448cdd8d8d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:41.197972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7164b65-b2b3-4e30-97f1-448cdd8d8d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:41.199596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7164b65-b2b3-4e30-97f1-448cdd8d8d53 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:41.211986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1269/2491] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:06:41.216453 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc97b7c-8da6-4bf0-b743-de08004191b9 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:41.226770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc97b7c-8da6-4bf0-b743-de08004191b9 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:41.233463 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3320ae-a580-4f45-8f91-740a3f8cb0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:41.234394 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3320ae-a580-4f45-8f91-740a3f8cb0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:41.234678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3320ae-a580-4f45-8f91-740a3f8cb0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:41.234963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3320ae-a580-4f45-8f91-740a3f8cb0f2 admin admin] PATH_INFO: `/v3/projects/b53809f583eb434188205a4b86271c6a/users/77da467b333c417393edb63e0334bdfd/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:41.251154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3320ae-a580-4f45-8f91-740a3f8cb0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b53809f583eb434188205a4b86271c6a, user_id=77da467b333c417393edb63e0334bdfd, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:41.253867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3320ae-a580-4f45-8f91-740a3f8cb0f2 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:41.361780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1223/2492] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:06:41 2022] PUT /identity/v3/projects/b53809f583eb434188205a4b86271c6a/users/77da467b333c417393edb63e0334bdfd/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:41.376985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb240de-541b-4075-9a07-9339da604c73 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:41.377313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb240de-541b-4075-9a07-9339da604c73 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:41.377602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb240de-541b-4075-9a07-9339da604c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:41.412579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-7fb240de-541b-4075-9a07-9339da604c73 None None] MFA Rules not processed for user `77da467b333c417393edb63e0334bdfd`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:41.413847 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7fb240de-541b-4075-9a07-9339da604c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:41.489280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1270/2493] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:41.502304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3320ae-a580-4f45-8f91-740a3f8cb0f2 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:41.504180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-af3320ae-a580-4f45-8f91-740a3f8cb0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:41.519999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:41.601891 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3320ae-a580-4f45-8f91-740a3f8cb0f2 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:41.609051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a649e26-c0b0-4dfc-8704-4198bbad55f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:41.609844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a649e26-c0b0-4dfc-8704-4198bbad55f6 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:41.610127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a649e26-c0b0-4dfc-8704-4198bbad55f6 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:41.610404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a649e26-c0b0-4dfc-8704-4198bbad55f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:41.622647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a649e26-c0b0-4dfc-8704-4198bbad55f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:41.625099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a649e26-c0b0-4dfc-8704-4198bbad55f6 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:41.637036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1224/2494] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:06:42.639590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb240de-541b-4075-9a07-9339da604c73 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:42.654341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb240de-541b-4075-9a07-9339da604c73 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:42.660171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:42.660875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:42.664106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:42.664106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:42.664106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:42.678280 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:42.681140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:42.696193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1271/2495] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:44.390140 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bb2b7-d26b-4f83-ae24-2d9834387b10 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:44.390820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bb2b7-d26b-4f83-ae24-2d9834387b10 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:44.390820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bb2b7-d26b-4f83-ae24-2d9834387b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:44.429133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ca8bb2b7-d26b-4f83-ae24-2d9834387b10 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:44.430857 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ca8bb2b7-d26b-4f83-ae24-2d9834387b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:44.498196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1225/2496] 10.222.0.17 () {58 vars in 967 bytes} [Sun Jul 24 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:44.702746 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:44.704572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:44.762129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:44.782818 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc65b6-3acc-41a8-9e34-7a26d87c090a service nova] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:44.788301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8ac66-e333-4a83-a309-5acaf63b1ffc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:44.797859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8ac66-e333-4a83-a309-5acaf63b1ffc service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:44.797859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8ac66-e333-4a83-a309-5acaf63b1ffc service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:44.797859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8ac66-e333-4a83-a309-5acaf63b1ffc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:44.799391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8ac66-e333-4a83-a309-5acaf63b1ffc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:44.801057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8ac66-e333-4a83-a309-5acaf63b1ffc service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:44.811278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1272/2497] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:06:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:06:48.344162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bb2b7-d26b-4f83-ae24-2d9834387b10 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:48.346392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ca8bb2b7-d26b-4f83-ae24-2d9834387b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:48.372852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bb2b7-d26b-4f83-ae24-2d9834387b10 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:48.383166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945a344-c03d-4b7c-92ab-570c5660c9af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:48.385099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945a344-c03d-4b7c-92ab-570c5660c9af service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:48.385922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945a344-c03d-4b7c-92ab-570c5660c9af service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:48.386647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945a344-c03d-4b7c-92ab-570c5660c9af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:48.390501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5945a344-c03d-4b7c-92ab-570c5660c9af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:48.508416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945a344-c03d-4b7c-92ab-570c5660c9af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:48.512789 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945a344-c03d-4b7c-92ab-570c5660c9af service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:48.535504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1226/2498] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:50.422380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee67794-134c-48f8-8bee-089c4f79f2d0 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:50.422380 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee67794-134c-48f8-8bee-089c4f79f2d0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:50.423071 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee67794-134c-48f8-8bee-089c4f79f2d0 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:50.424098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1273/2499] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:06:50.440556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e241fce-25c6-4316-8d53-6bbbbcc0c765 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:50.441166 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e241fce-25c6-4316-8d53-6bbbbcc0c765 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:50.441289 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e241fce-25c6-4316-8d53-6bbbbcc0c765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:50.462600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-5e241fce-25c6-4316-8d53-6bbbbcc0c765 None None] MFA Rules not processed for user `1c1b23b83544423abc6919d805916b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:50.464347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5e241fce-25c6-4316-8d53-6bbbbcc0c765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:50.509164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:50.534732 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1227/2500] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:06:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:06:50.547145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee67794-134c-48f8-8bee-089c4f79f2d0 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:50.549004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6ee67794-134c-48f8-8bee-089c4f79f2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:50.573730 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee67794-134c-48f8-8bee-089c4f79f2d0 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:50.579248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b9a49-de2c-4990-b331-22d33f31c39c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:50.579926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b9a49-de2c-4990-b331-22d33f31c39c service placement] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:50.580216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b9a49-de2c-4990-b331-22d33f31c39c service placement] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:50.580501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b9a49-de2c-4990-b331-22d33f31c39c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:50.590536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b9a49-de2c-4990-b331-22d33f31c39c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:50.592207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b9a49-de2c-4990-b331-22d33f31c39c service placement] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:50.606043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1274/2501] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:51.573427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e241fce-25c6-4316-8d53-6bbbbcc0c765 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:51.638832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e241fce-25c6-4316-8d53-6bbbbcc0c765 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:51.643874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:51.644913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:51.645203 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:51.645670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] PATH_INFO: `/v3/users/77da467b333c417393edb63e0334bdfd` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:51.646909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77da467b333c417393edb63e0334bdfd)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:51.648934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:51.941493 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77da467b333c417393edb63e0334bdfd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:51.952021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77da467b333c417393edb63e0334bdfd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:51.952407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77da467b333c417393edb63e0334bdfd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:52.030495 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b9a49-de2c-4990-b331-22d33f31c39c service placement] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:52.032652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-5e5b9a49-de2c-4990-b331-22d33f31c39c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:52.102386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77da467b333c417393edb63e0334bdfd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:52.105511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b9a49-de2c-4990-b331-22d33f31c39c service placement] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:52.109987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:52.111078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:52.111544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:52.111952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:52.114258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:52.124712 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-689c785d-88be-465e-b673-dba7b7d9fdf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77da467b333c417393edb63e0334bdfd'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:52.135937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1228/2502] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:06:51 2022] DELETE /identity/v3/users/77da467b333c417393edb63e0334bdfd => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:52.143751 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b858d8-48d7-4925-a394-34831069e8e5 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:52.144704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b858d8-48d7-4925-a394-34831069e8e5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:52.145191 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b858d8-48d7-4925-a394-34831069e8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:52.179690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-20b858d8-48d7-4925-a394-34831069e8e5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:52.181099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-20b858d8-48d7-4925-a394-34831069e8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:52.200232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:52.201845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:52.214806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1275/2503] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:06:52.252097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:52.264979 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1229/2504] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:52.278232 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:52.280125 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:52.292237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07158b-b1cb-4f37-8657-155cf63e9f15 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:52.357893 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a66131-996f-4464-b879-1ed10f2398e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:52.358725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a66131-996f-4464-b879-1ed10f2398e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:52.359243 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a66131-996f-4464-b879-1ed10f2398e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:52.359648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a66131-996f-4464-b879-1ed10f2398e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:52.371229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a66131-996f-4464-b879-1ed10f2398e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:52.372408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a66131-996f-4464-b879-1ed10f2398e6 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:52.388244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1276/2505] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:53.442035 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b858d8-48d7-4925-a394-34831069e8e5 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:53.443101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-20b858d8-48d7-4925-a394-34831069e8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:53.468403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b858d8-48d7-4925-a394-34831069e8e5 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:53.485423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:53.487051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:53.487051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:53.487051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] PATH_INFO: `/v3/projects/b53809f583eb434188205a4b86271c6a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:53.488458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b53809f583eb434188205a4b86271c6a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:53.490523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:53.656555 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b53809f583eb434188205a4b86271c6a', 'request_id': 'req-6d341234-e307-40de-84ef-618457430f80'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:53.670976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b53809f583eb434188205a4b86271c6a', 'request_id': 'req-6d341234-e307-40de-84ef-618457430f80'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:53.764265 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b53809f583eb434188205a4b86271c6a', 'request_id': 'req-6d341234-e307-40de-84ef-618457430f80'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:53.787488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] The token cache is being invalidate because project b53809f583eb434188205a4b86271c6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:06:53.788484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:53.790142 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-6d341234-e307-40de-84ef-618457430f80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:53.793355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1230/2506] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:06:53 2022] DELETE /identity/v3/projects/b53809f583eb434188205a4b86271c6a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:53.801435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26da05-7846-44c9-b05d-69da9b7440fb None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:53.802920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26da05-7846-44c9-b05d-69da9b7440fb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:53.803408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26da05-7846-44c9-b05d-69da9b7440fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:53.836345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-3a26da05-7846-44c9-b05d-69da9b7440fb None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:53.837678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3a26da05-7846-44c9-b05d-69da9b7440fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:53.853693 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:53.920774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1277/2507] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:53.928597 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56f04-fb4b-4fc5-a011-5d993238b38e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:53.928948 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56f04-fb4b-4fc5-a011-5d993238b38e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:53.929272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56f04-fb4b-4fc5-a011-5d993238b38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:53.954406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-13c56f04-fb4b-4fc5-a011-5d993238b38e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:53.956407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-13c56f04-fb4b-4fc5-a011-5d993238b38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:53.966178 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1231/2508] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:06:53.972739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26da05-7846-44c9-b05d-69da9b7440fb None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:54.028404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26da05-7846-44c9-b05d-69da9b7440fb None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:54.034958 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f83253-4e0f-43b8-a6ce-010a0d886199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:54.036161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f83253-4e0f-43b8-a6ce-010a0d886199 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:54.036794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f83253-4e0f-43b8-a6ce-010a0d886199 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:54.037294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f83253-4e0f-43b8-a6ce-010a0d886199 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:54.038491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f83253-4e0f-43b8-a6ce-010a0d886199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:06:54.039302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f83253-4e0f-43b8-a6ce-010a0d886199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:54.041351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f83253-4e0f-43b8-a6ce-010a0d886199 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:54.064813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1278/2509] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:06:54.067332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c56f04-fb4b-4fc5-a011-5d993238b38e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:54.078704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c56f04-fb4b-4fc5-a011-5d993238b38e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:54.086634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f934b-33a5-46f3-bde1-f7c251160236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:54.087294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f934b-33a5-46f3-bde1-f7c251160236 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:54.087690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f934b-33a5-46f3-bde1-f7c251160236 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:54.087975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f934b-33a5-46f3-bde1-f7c251160236 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:54.088745 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f934b-33a5-46f3-bde1-f7c251160236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:54.090935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f934b-33a5-46f3-bde1-f7c251160236 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:54.213225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1232/2510] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:06:54 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:06:54.220007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f83253-4e0f-43b8-a6ce-010a0d886199 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:54.239117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f83253-4e0f-43b8-a6ce-010a0d886199 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:54.315687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153f9ec-d579-4462-a53a-c5e6fa2c8dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:54.316470 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153f9ec-d579-4462-a53a-c5e6fa2c8dcc admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:54.316856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153f9ec-d579-4462-a53a-c5e6fa2c8dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:54.317207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153f9ec-d579-4462-a53a-c5e6fa2c8dcc admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:54.317993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153f9ec-d579-4462-a53a-c5e6fa2c8dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:54.320058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153f9ec-d579-4462-a53a-c5e6fa2c8dcc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:54.441600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1279/2511] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:06:54 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:06:54.449717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f934b-33a5-46f3-bde1-f7c251160236 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:54.477952 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f934b-33a5-46f3-bde1-f7c251160236 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:54.487457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c46a7-a24a-40ce-9fc9-e8ba22f1f2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:54.488707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c46a7-a24a-40ce-9fc9-e8ba22f1f2cf admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:54.489249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c46a7-a24a-40ce-9fc9-e8ba22f1f2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:54.490108 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c46a7-a24a-40ce-9fc9-e8ba22f1f2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:54.492126 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c46a7-a24a-40ce-9fc9-e8ba22f1f2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:54.500382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c46a7-a24a-40ce-9fc9-e8ba22f1f2cf admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:54.529217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1233/2512] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:06:54.539035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153f9ec-d579-4462-a53a-c5e6fa2c8dcc admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:54.562523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153f9ec-d579-4462-a53a-c5e6fa2c8dcc admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:54.571337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d0e2b-d2bf-46f9-9d6b-3953b081b346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:54.572881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d0e2b-d2bf-46f9-9d6b-3953b081b346 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:54.574066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d0e2b-d2bf-46f9-9d6b-3953b081b346 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:54.574882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d0e2b-d2bf-46f9-9d6b-3953b081b346 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:54.576636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d0e2b-d2bf-46f9-9d6b-3953b081b346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:54.580097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d0e2b-d2bf-46f9-9d6b-3953b081b346 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:54.613086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1280/2513] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:06:54.620198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c46a7-a24a-40ce-9fc9-e8ba22f1f2cf admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:54.657726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c46a7-a24a-40ce-9fc9-e8ba22f1f2cf admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:54.663753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f3ef8-9ef0-4b18-aed1-691f8742a29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:54.664808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f3ef8-9ef0-4b18-aed1-691f8742a29b admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:54.665099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f3ef8-9ef0-4b18-aed1-691f8742a29b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:54.665521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f3ef8-9ef0-4b18-aed1-691f8742a29b admin admin] PATH_INFO: `/v3/projects/856f757c01f54d30871f3c56ec3fcff3/users/2a417f22d1cb40098e72a483f2684631/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:54.689889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f3ef8-9ef0-4b18-aed1-691f8742a29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=856f757c01f54d30871f3c56ec3fcff3, user_id=2a417f22d1cb40098e72a483f2684631, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:54.696304 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f3ef8-9ef0-4b18-aed1-691f8742a29b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:54.806129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1234/2514] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:06:54 2022] PUT /identity/v3/projects/856f757c01f54d30871f3c56ec3fcff3/users/2a417f22d1cb40098e72a483f2684631/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:54.818992 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ce90d-025a-4019-b753-e914c0c767e2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:54.819996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ce90d-025a-4019-b753-e914c0c767e2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:54.819996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ce90d-025a-4019-b753-e914c0c767e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:54.857263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a41ce90d-025a-4019-b753-e914c0c767e2 None None] MFA Rules not processed for user `2a417f22d1cb40098e72a483f2684631`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:54.858890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a41ce90d-025a-4019-b753-e914c0c767e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:54.953878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1281/2515] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:54.969831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f3ef8-9ef0-4b18-aed1-691f8742a29b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:54.972024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-886f3ef8-9ef0-4b18-aed1-691f8742a29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:55.074091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f3ef8-9ef0-4b18-aed1-691f8742a29b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:55.080829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c241f51-fb2e-45fd-a2de-1086783141ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:55.081700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c241f51-fb2e-45fd-a2de-1086783141ab service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:55.082093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c241f51-fb2e-45fd-a2de-1086783141ab service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:55.082474 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c241f51-fb2e-45fd-a2de-1086783141ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:55.104440 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c241f51-fb2e-45fd-a2de-1086783141ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:55.108163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c241f51-fb2e-45fd-a2de-1086783141ab service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:55.134665 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1235/2516] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:55.495150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ce90d-025a-4019-b753-e914c0c767e2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:55.514007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:55.620864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ce90d-025a-4019-b753-e914c0c767e2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:55.630022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:55.631115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:55.631638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:55.631903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] PATH_INFO: `/v3/users/2a417f22d1cb40098e72a483f2684631` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:55.633296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a417f22d1cb40098e72a483f2684631)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:55.635377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:55.777340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c241f51-fb2e-45fd-a2de-1086783141ab service nova] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:55.777340 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0c241f51-fb2e-45fd-a2de-1086783141ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:55.790294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:55.799836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c241f51-fb2e-45fd-a2de-1086783141ab service nova] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:55.811369 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2e439-9bcc-4904-b253-ac010d71a300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:55.812623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2e439-9bcc-4904-b253-ac010d71a300 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:55.813104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2e439-9bcc-4904-b253-ac010d71a300 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:55.814147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2e439-9bcc-4904-b253-ac010d71a300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:55.818404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6ed2e439-9bcc-4904-b253-ac010d71a300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:55.844060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2e439-9bcc-4904-b253-ac010d71a300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:55.846098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2e439-9bcc-4904-b253-ac010d71a300 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:55.859947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1236/2517] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:55.976825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a417f22d1cb40098e72a483f2684631'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:56.006505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a417f22d1cb40098e72a483f2684631'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:56.023430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a417f22d1cb40098e72a483f2684631'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:56.024663 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a417f22d1cb40098e72a483f2684631'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:56.138517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a417f22d1cb40098e72a483f2684631'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:56.148578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1282/2518] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:06:55 2022] DELETE /identity/v3/users/2a417f22d1cb40098e72a483f2684631 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:56.154771 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca04e52-23cd-4e9e-8182-50d8605f25e3 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:56.155150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca04e52-23cd-4e9e-8182-50d8605f25e3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:56.155492 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca04e52-23cd-4e9e-8182-50d8605f25e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:56.179925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-eca04e52-23cd-4e9e-8182-50d8605f25e3 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:56.182843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-eca04e52-23cd-4e9e-8182-50d8605f25e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:56.273486 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1237/2519] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:06:56.284521 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:56.287153 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:56.370682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd96de-38f7-4abe-92d7-505f5314dc57 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:56.376796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54075f-8f3c-4712-a6a7-148bd281ece6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:56.377865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54075f-8f3c-4712-a6a7-148bd281ece6 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:56.378369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54075f-8f3c-4712-a6a7-148bd281ece6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:56.378849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54075f-8f3c-4712-a6a7-148bd281ece6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:56.394117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54075f-8f3c-4712-a6a7-148bd281ece6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:56.397505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54075f-8f3c-4712-a6a7-148bd281ece6 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:56.424904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1283/2520] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:06:56.933870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca04e52-23cd-4e9e-8182-50d8605f25e3 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:56.972590 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-eca04e52-23cd-4e9e-8182-50d8605f25e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:57.001601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca04e52-23cd-4e9e-8182-50d8605f25e3 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:57.007244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:57.008521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:57.009056 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:57.009603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] PATH_INFO: `/v3/users/3f1b6145d67d4cbf9dcd398736e6b0da` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:57.012283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f1b6145d67d4cbf9dcd398736e6b0da)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:57.015709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:57.279842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f1b6145d67d4cbf9dcd398736e6b0da'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.300716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f1b6145d67d4cbf9dcd398736e6b0da'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.302688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f1b6145d67d4cbf9dcd398736e6b0da'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.312440 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54075f-8f3c-4712-a6a7-148bd281ece6 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:57.314650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1c54075f-8f3c-4712-a6a7-148bd281ece6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:57.334993 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54075f-8f3c-4712-a6a7-148bd281ece6 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:57.348455 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:57.349685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:57.350165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:57.350614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] PATH_INFO: `/v3/projects/856f757c01f54d30871f3c56ec3fcff3` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:57.352385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=856f757c01f54d30871f3c56ec3fcff3)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:57.355789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:57.412620 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f1b6145d67d4cbf9dcd398736e6b0da'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.434881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-93677a5d-0e7a-43cd-bc00-51a7e94abaea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f1b6145d67d4cbf9dcd398736e6b0da'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.448075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1238/2521] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:06:56 2022] DELETE /identity/v3/users/3f1b6145d67d4cbf9dcd398736e6b0da => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:57.493743 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:57.524462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '856f757c01f54d30871f3c56ec3fcff3', 'request_id': 'req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.531846 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '856f757c01f54d30871f3c56ec3fcff3', 'request_id': 'req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.540187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '856f757c01f54d30871f3c56ec3fcff3', 'request_id': 'req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.616826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] The token cache is being invalidate because project 856f757c01f54d30871f3c56ec3fcff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:06:57.621827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.621827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ad6ae5d5-49d1-48f7-b272-6cf02ee0fb0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:57.621827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1284/2522] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:06:57 2022] DELETE /identity/v3/projects/856f757c01f54d30871f3c56ec3fcff3 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:57.626356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec09306-e124-4acd-8e5a-7901994ebf97 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:57.626691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec09306-e124-4acd-8e5a-7901994ebf97 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:57.626910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec09306-e124-4acd-8e5a-7901994ebf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:57.659080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-7ec09306-e124-4acd-8e5a-7901994ebf97 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:57.660980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7ec09306-e124-4acd-8e5a-7901994ebf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:57.739530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1239/2523] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:57.749265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f69d3-9994-4458-87ff-d01977f6be9d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:57.749532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f69d3-9994-4458-87ff-d01977f6be9d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:57.749737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f69d3-9994-4458-87ff-d01977f6be9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:57.769106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1e3f69d3-9994-4458-87ff-d01977f6be9d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:57.770420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1e3f69d3-9994-4458-87ff-d01977f6be9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:57.776359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec09306-e124-4acd-8e5a-7901994ebf97 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:57.777836 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7ec09306-e124-4acd-8e5a-7901994ebf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:57.780337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1285/2524] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:57.785203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f69d3-9994-4458-87ff-d01977f6be9d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:57.788376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:57.801851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec09306-e124-4acd-8e5a-7901994ebf97 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:57.802189 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f69d3-9994-4458-87ff-d01977f6be9d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:57.809938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1dd83-231e-4c15-b44a-555f3cb88c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:57.810502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1dd83-231e-4c15-b44a-555f3cb88c76 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:57.810849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1dd83-231e-4c15-b44a-555f3cb88c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:57.811138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1dd83-231e-4c15-b44a-555f3cb88c76 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:57.811865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1dd83-231e-4c15-b44a-555f3cb88c76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:06:57.812171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1dd83-231e-4c15-b44a-555f3cb88c76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:57.813897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1dd83-231e-4c15-b44a-555f3cb88c76 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:57.815988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:57.816959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:57.817232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:57.817524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] PATH_INFO: `/v3/projects/4d31f280cfc640dd8381ee0869d9f455` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:57.818889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d31f280cfc640dd8381ee0869d9f455)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:57.820968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:57.836623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1286/2525] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:06:57.841704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1dd83-231e-4c15-b44a-555f3cb88c76 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:57.853385 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1dd83-231e-4c15-b44a-555f3cb88c76 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:57.859033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c0f0b-fbbe-415d-8e05-f09663b0fdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:57.859713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c0f0b-fbbe-415d-8e05-f09663b0fdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:57.860000 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c0f0b-fbbe-415d-8e05-f09663b0fdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:57.860276 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c0f0b-fbbe-415d-8e05-f09663b0fdc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:57.861039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c0f0b-fbbe-415d-8e05-f09663b0fdc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:57.863003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c0f0b-fbbe-415d-8e05-f09663b0fdc0 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:58.029814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1287/2526] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:06:57 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:06:58.030962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c0f0b-fbbe-415d-8e05-f09663b0fdc0 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:58.038208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d31f280cfc640dd8381ee0869d9f455', 'request_id': 'req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.045620 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d31f280cfc640dd8381ee0869d9f455', 'request_id': 'req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.095029 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c0f0b-fbbe-415d-8e05-f09663b0fdc0 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:58.102255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358dd164-87c8-4cce-8ef8-fb2889db15c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:58.102952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dd164-87c8-4cce-8ef8-fb2889db15c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:58.103316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dd164-87c8-4cce-8ef8-fb2889db15c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:58.103574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dd164-87c8-4cce-8ef8-fb2889db15c2 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:58.104368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358dd164-87c8-4cce-8ef8-fb2889db15c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:58.106871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358dd164-87c8-4cce-8ef8-fb2889db15c2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:58.158480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d31f280cfc640dd8381ee0869d9f455', 'request_id': 'req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.170365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] The token cache is being invalidate because project 4d31f280cfc640dd8381ee0869d9f455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:06:58.170659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.171300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.173068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1240/2527] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:06:57 2022] DELETE /identity/v3/projects/4d31f280cfc640dd8381ee0869d9f455 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:06:58.176939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:58.178297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:58.220161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1288/2528] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:06:58 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:06:58.224391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358dd164-87c8-4cce-8ef8-fb2889db15c2 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:58.229097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-358dd164-87c8-4cce-8ef8-fb2889db15c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:58.240092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358dd164-87c8-4cce-8ef8-fb2889db15c2 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:58.245028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedcf17-d995-4fba-8018-3a2bc440a765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:58.245643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedcf17-d995-4fba-8018-3a2bc440a765 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:58.245932 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedcf17-d995-4fba-8018-3a2bc440a765 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:58.246211 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedcf17-d995-4fba-8018-3a2bc440a765 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:58.246959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedcf17-d995-4fba-8018-3a2bc440a765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:58.248229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0697f64-78b5-4f07-8f3a-bb8fa0e8826a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:58.248600 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedcf17-d995-4fba-8018-3a2bc440a765 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:58.253511 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:58.254402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:58.254686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:58.255026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] PATH_INFO: `/v3/users/019c27e5d7564e82a2c666cbc87c378e` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:58.256257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=019c27e5d7564e82a2c666cbc87c378e)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:58.258228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:58.263908 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1289/2529] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:06:58.265956 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedcf17-d995-4fba-8018-3a2bc440a765 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:58.278021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedcf17-d995-4fba-8018-3a2bc440a765 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:58.284127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455e5f9-6270-4ff2-a092-5342edfaa967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:58.285354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455e5f9-6270-4ff2-a092-5342edfaa967 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:58.285354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455e5f9-6270-4ff2-a092-5342edfaa967 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:58.285354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455e5f9-6270-4ff2-a092-5342edfaa967 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:58.286032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455e5f9-6270-4ff2-a092-5342edfaa967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:58.287672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455e5f9-6270-4ff2-a092-5342edfaa967 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:58.301333 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1290/2530] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:06:58.305812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455e5f9-6270-4ff2-a092-5342edfaa967 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:58.317108 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455e5f9-6270-4ff2-a092-5342edfaa967 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:58.322245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31e601-d16b-4455-88d6-90c75a1d6dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:58.323728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31e601-d16b-4455-88d6-90c75a1d6dba admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:58.324021 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31e601-d16b-4455-88d6-90c75a1d6dba admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:58.324301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31e601-d16b-4455-88d6-90c75a1d6dba admin admin] PATH_INFO: `/v3/projects/a1a2da7d58ad4d1da9830b418e23be73/users/baaf249db83042758e839a64f17b4cd7/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:58.343009 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31e601-d16b-4455-88d6-90c75a1d6dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a2da7d58ad4d1da9830b418e23be73, user_id=baaf249db83042758e839a64f17b4cd7, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:58.345795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31e601-d16b-4455-88d6-90c75a1d6dba admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:58.430207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1291/2531] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:06:58 2022] PUT /identity/v3/projects/a1a2da7d58ad4d1da9830b418e23be73/users/baaf249db83042758e839a64f17b4cd7/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:06:58.444288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcbca7-5195-4259-80d5-041e6ab8e1bd None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:58.444621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcbca7-5195-4259-80d5-041e6ab8e1bd None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:58.444898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcbca7-5195-4259-80d5-041e6ab8e1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:58.477233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-59fcbca7-5195-4259-80d5-041e6ab8e1bd None None] MFA Rules not processed for user `baaf249db83042758e839a64f17b4cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:58.479327 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-59fcbca7-5195-4259-80d5-041e6ab8e1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:58.494434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:58.533626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019c27e5d7564e82a2c666cbc87c378e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.547144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019c27e5d7564e82a2c666cbc87c378e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.548407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019c27e5d7564e82a2c666cbc87c378e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.572441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1292/2532] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:06:58.578466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcbca7-5195-4259-80d5-041e6ab8e1bd None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:58.650630 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcbca7-5195-4259-80d5-041e6ab8e1bd None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:58.655988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a517d7f-d857-48ea-abcc-d33a423f3862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:58.656698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a517d7f-d857-48ea-abcc-d33a423f3862 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:58.656988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a517d7f-d857-48ea-abcc-d33a423f3862 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:58.657268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a517d7f-d857-48ea-abcc-d33a423f3862 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:58.657987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a517d7f-d857-48ea-abcc-d33a423f3862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:58.659927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a517d7f-d857-48ea-abcc-d33a423f3862 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:58.685816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019c27e5d7564e82a2c666cbc87c378e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.700970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019c27e5d7564e82a2c666cbc87c378e'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:58.708248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1241/2533] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:06:58 2022] DELETE /identity/v3/users/019c27e5d7564e82a2c666cbc87c378e => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:58.804881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1293/2534] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:06:58 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:06:58.809225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:58.810840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:58.817776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:58.887882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ceded-db63-4bd0-97d7-f5070a862de4 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:58.966357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0eb5b-345e-46fe-a05b-9bfbfdb86acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:58.967052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0eb5b-345e-46fe-a05b-9bfbfdb86acb admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:58.967358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0eb5b-345e-46fe-a05b-9bfbfdb86acb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:58.967633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0eb5b-345e-46fe-a05b-9bfbfdb86acb admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:58.968417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0eb5b-345e-46fe-a05b-9bfbfdb86acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:58.970560 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0eb5b-345e-46fe-a05b-9bfbfdb86acb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:59.064244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a517d7f-d857-48ea-abcc-d33a423f3862 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:59.066225 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1a517d7f-d857-48ea-abcc-d33a423f3862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:59.081945 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a517d7f-d857-48ea-abcc-d33a423f3862 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:59.088289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:59.094918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:59.094918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:59.094918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] PATH_INFO: `/v3/projects/63a05ee2b3fe48ddbac52390d653027a` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:59.094918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a05ee2b3fe48ddbac52390d653027a)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:59.094918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:59.101657 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1242/2535] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:06:58 2022] POST /identity/v3/users => generated 296 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:06:59.105614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0eb5b-345e-46fe-a05b-9bfbfdb86acb admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:59.119068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0eb5b-345e-46fe-a05b-9bfbfdb86acb admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:59.125730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f371a6-a4a8-447b-99f4-00585892295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:59.126342 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f371a6-a4a8-447b-99f4-00585892295b admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:59.126670 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f371a6-a4a8-447b-99f4-00585892295b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:59.127005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f371a6-a4a8-447b-99f4-00585892295b admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:59.127715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f371a6-a4a8-447b-99f4-00585892295b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:59.129356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f371a6-a4a8-447b-99f4-00585892295b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:59.141208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1243/2536] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:06:59.147703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f371a6-a4a8-447b-99f4-00585892295b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:59.158577 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f371a6-a4a8-447b-99f4-00585892295b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:59.165695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f4caf-82b5-466b-b068-0cab87fbf940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:59.165695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f4caf-82b5-466b-b068-0cab87fbf940 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:59.165695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f4caf-82b5-466b-b068-0cab87fbf940 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:59.165695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f4caf-82b5-466b-b068-0cab87fbf940 admin admin] PATH_INFO: `/v3/projects/e104fcbffb284649a2162c0b52a3c5db/users/09e95f6d0ffc42ed8da1c245e2e05162/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:59.181368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f4caf-82b5-466b-b068-0cab87fbf940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e104fcbffb284649a2162c0b52a3c5db, user_id=09e95f6d0ffc42ed8da1c245e2e05162, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:59.184373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f4caf-82b5-466b-b068-0cab87fbf940 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:59.329881 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1244/2537] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:06:59 2022] PUT /identity/v3/projects/e104fcbffb284649a2162c0b52a3c5db/users/09e95f6d0ffc42ed8da1c245e2e05162/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:06:59.334793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f4caf-82b5-466b-b068-0cab87fbf940 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:59.340719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a05ee2b3fe48ddbac52390d653027a', 'request_id': 'req-ec775379-c018-42c7-9d14-4953884f19aa'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:59.347880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a05ee2b3fe48ddbac52390d653027a', 'request_id': 'req-ec775379-c018-42c7-9d14-4953884f19aa'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:59.357886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a05ee2b3fe48ddbac52390d653027a', 'request_id': 'req-ec775379-c018-42c7-9d14-4953884f19aa'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:59.410859 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f4caf-82b5-466b-b068-0cab87fbf940 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:59.421999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:59.468794 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] The token cache is being invalidate because project 63a05ee2b3fe48ddbac52390d653027a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:06:59.469115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:59.469862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:06:59.471500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1294/2538] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:06:59 2022] DELETE /identity/v3/projects/63a05ee2b3fe48ddbac52390d653027a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:06:59.475877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22742-fc47-4aea-ae57-1f46aa6feb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:59.476788 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22742-fc47-4aea-ae57-1f46aa6feb71 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:59.477501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22742-fc47-4aea-ae57-1f46aa6feb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:59.478118 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22742-fc47-4aea-ae57-1f46aa6feb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:59.479023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22742-fc47-4aea-ae57-1f46aa6feb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:59.481465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22742-fc47-4aea-ae57-1f46aa6feb71 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:59.496233 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1245/2539] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:06:59.501126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:59.502639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:59.518098 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec775379-c018-42c7-9d14-4953884f19aa admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:59.524573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6ab4c-26ad-4a4d-b56d-f1bd92db7dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:59.525543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ab4c-26ad-4a4d-b56d-f1bd92db7dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:59.525827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ab4c-26ad-4a4d-b56d-f1bd92db7dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:59.526109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ab4c-26ad-4a4d-b56d-f1bd92db7dcc admin admin] PATH_INFO: `/v3/projects/e104fcbffb284649a2162c0b52a3c5db/users/09e95f6d0ffc42ed8da1c245e2e05162/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:59.528971 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6ab4c-26ad-4a4d-b56d-f1bd92db7dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e104fcbffb284649a2162c0b52a3c5db, user_id=09e95f6d0ffc42ed8da1c245e2e05162, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:59.531736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6ab4c-26ad-4a4d-b56d-f1bd92db7dcc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:59.547205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1295/2540] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:06:59 2022] PUT /identity/v3/projects/e104fcbffb284649a2162c0b52a3c5db/users/09e95f6d0ffc42ed8da1c245e2e05162/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:06:59.561643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66003fb-c529-46b5-9807-2a3e595aa2fd None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:59.562194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66003fb-c529-46b5-9807-2a3e595aa2fd None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:59.567306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66003fb-c529-46b5-9807-2a3e595aa2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:59.617185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d66003fb-c529-46b5-9807-2a3e595aa2fd None None] MFA Rules not processed for user `09e95f6d0ffc42ed8da1c245e2e05162`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:06:59.619017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d66003fb-c529-46b5-9807-2a3e595aa2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:59.772414 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1246/2541] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:06:59.781139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6ab4c-26ad-4a4d-b56d-f1bd92db7dcc admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:06:59.783148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e0c6ab4c-26ad-4a4d-b56d-f1bd92db7dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:06:59.790520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:06:59.860099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6ab4c-26ad-4a4d-b56d-f1bd92db7dcc admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:06:59.866059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6442a-a723-4a78-8dde-0fe8d9cdb71e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:06:59.867250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6442a-a723-4a78-8dde-0fe8d9cdb71e service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:06:59.867784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6442a-a723-4a78-8dde-0fe8d9cdb71e service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:06:59.868302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6442a-a723-4a78-8dde-0fe8d9cdb71e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:06:59.883568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6442a-a723-4a78-8dde-0fe8d9cdb71e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:06:59.885075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6442a-a723-4a78-8dde-0fe8d9cdb71e service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:06:59.895516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1296/2542] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:06:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:00.124831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66003fb-c529-46b5-9807-2a3e595aa2fd None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:00.135146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66003fb-c529-46b5-9807-2a3e595aa2fd None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:00.138817 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc977e-e933-49da-998a-eca5e372c932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:00.139501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc977e-e933-49da-998a-eca5e372c932 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:00.143701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc977e-e933-49da-998a-eca5e372c932 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:00.143701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc977e-e933-49da-998a-eca5e372c932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:00.143701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-dabc977e-e933-49da-998a-eca5e372c932 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:00.210021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc977e-e933-49da-998a-eca5e372c932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:00.211484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc977e-e933-49da-998a-eca5e372c932 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:00.221970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1247/2543] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:07.128991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6442a-a723-4a78-8dde-0fe8d9cdb71e service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:07.131044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e5c6442a-a723-4a78-8dde-0fe8d9cdb71e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:07.219507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6442a-a723-4a78-8dde-0fe8d9cdb71e service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:07.225331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:07.225997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:07.226287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:07.226596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:07.228480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:07.242415 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:07.244215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:07.262017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1297/2544] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:08.743472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2835b-307c-49ed-a996-1bf639e30531 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:08.744112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2835b-307c-49ed-a996-1bf639e30531 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:08.744112 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2835b-307c-49ed-a996-1bf639e30531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:08.763853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-93a2835b-307c-49ed-a996-1bf639e30531 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:08.766180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-93a2835b-307c-49ed-a996-1bf639e30531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:08.800426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:08.841870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1248/2545] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:07:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:08.860843 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:08.862725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:08.967489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb7941-8d46-4643-88ca-ee15dd117ca0 service nova] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:08.975994 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79963019-ec4d-4005-9f94-b5708b317372 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:08.976906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79963019-ec4d-4005-9f94-b5708b317372 service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:08.977335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79963019-ec4d-4005-9f94-b5708b317372 service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:08.977840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79963019-ec4d-4005-9f94-b5708b317372 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:08.989364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79963019-ec4d-4005-9f94-b5708b317372 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:08.991298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79963019-ec4d-4005-9f94-b5708b317372 service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:09.019157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1298/2546] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:07:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 24 20:07:14.287011 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2835b-307c-49ed-a996-1bf639e30531 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:14.422269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2835b-307c-49ed-a996-1bf639e30531 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:14.434144 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:14.435617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:14.435617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:14.435827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] PATH_INFO: `/v3/users/baaf249db83042758e839a64f17b4cd7` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:14.437185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=baaf249db83042758e839a64f17b4cd7)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:14.440792 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:14.674249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baaf249db83042758e839a64f17b4cd7'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:14.694101 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baaf249db83042758e839a64f17b4cd7'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:14.695402 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baaf249db83042758e839a64f17b4cd7'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:14.821529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baaf249db83042758e839a64f17b4cd7'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:14.858582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baaf249db83042758e839a64f17b4cd7'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:14.872359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1249/2547] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:07:14 2022] DELETE /identity/v3/users/baaf249db83042758e839a64f17b4cd7 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:14.879501 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e1afc-fc63-404b-aa75-d1a53959c07a None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:14.881447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e1afc-fc63-404b-aa75-d1a53959c07a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:14.882036 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e1afc-fc63-404b-aa75-d1a53959c07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:14.915045 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-0b2e1afc-fc63-404b-aa75-d1a53959c07a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:14.918381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0b2e1afc-fc63-404b-aa75-d1a53959c07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:14.962760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:15.041356 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1299/2548] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:15.057418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:15.059363 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:15.180886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b63448-3b1d-4d7f-9cfb-f8f514b9fb1b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:15.187084 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a221a9-00ae-46da-8ac6-649415287fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:15.188309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a221a9-00ae-46da-8ac6-649415287fdf service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:15.188865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a221a9-00ae-46da-8ac6-649415287fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:15.189209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a221a9-00ae-46da-8ac6-649415287fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:15.212275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a221a9-00ae-46da-8ac6-649415287fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:15.216530 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a221a9-00ae-46da-8ac6-649415287fdf service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:15.235612 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:15.252951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1250/2549] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:07:15.930882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e1afc-fc63-404b-aa75-d1a53959c07a None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:15.933842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0b2e1afc-fc63-404b-aa75-d1a53959c07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:15.959975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e1afc-fc63-404b-aa75-d1a53959c07a None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:15.975446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:15.983587 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:15.984052 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:15.984620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] PATH_INFO: `/v3/projects/a1a2da7d58ad4d1da9830b418e23be73` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:15.987907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a2da7d58ad4d1da9830b418e23be73)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:15.991810 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:16.216047 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a2da7d58ad4d1da9830b418e23be73', 'request_id': 'req-f82066c5-524b-4d6a-9769-6f24bbef9cc6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:16.264164 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a2da7d58ad4d1da9830b418e23be73', 'request_id': 'req-f82066c5-524b-4d6a-9769-6f24bbef9cc6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:16.296325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a2da7d58ad4d1da9830b418e23be73', 'request_id': 'req-f82066c5-524b-4d6a-9769-6f24bbef9cc6'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:16.403736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] The token cache is being invalidate because project a1a2da7d58ad4d1da9830b418e23be73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:07:16.404049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:16.404848 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:16.406876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1300/2550] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:07:15 2022] DELETE /identity/v3/projects/a1a2da7d58ad4d1da9830b418e23be73 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:07:16.414213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a221a9-00ae-46da-8ac6-649415287fdf service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:16.442870 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a221a9-00ae-46da-8ac6-649415287fdf service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:16.537044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:16.538188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:16.538638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:16.539038 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] PATH_INFO: `/v3/users/09e95f6d0ffc42ed8da1c245e2e05162` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:16.540822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e95f6d0ffc42ed8da1c245e2e05162)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:16.544199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:17.210586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e95f6d0ffc42ed8da1c245e2e05162'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:17.222368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e95f6d0ffc42ed8da1c245e2e05162'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:17.224456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e95f6d0ffc42ed8da1c245e2e05162'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:17.356278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e95f6d0ffc42ed8da1c245e2e05162'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:17.399360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-a6f31de2-fc64-4316-9b43-f4594e7e88b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e95f6d0ffc42ed8da1c245e2e05162'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:17.424821 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1251/2551] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:07:16 2022] DELETE /identity/v3/users/09e95f6d0ffc42ed8da1c245e2e05162 => generated 0 bytes in 1011 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:18.246519 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:18.249611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:18.277768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82066c5-524b-4d6a-9769-6f24bbef9cc6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:18.414543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:18.415611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:18.415966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:18.416319 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] PATH_INFO: `/v3/projects/e104fcbffb284649a2162c0b52a3c5db` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:18.418628 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e104fcbffb284649a2162c0b52a3c5db)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:18.422319 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:18.630803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:18.633407 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e104fcbffb284649a2162c0b52a3c5db', 'request_id': 'req-90df913f-6133-4514-9a87-78df70a61d1d'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:18.642976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e104fcbffb284649a2162c0b52a3c5db', 'request_id': 'req-90df913f-6133-4514-9a87-78df70a61d1d'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:18.652776 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e104fcbffb284649a2162c0b52a3c5db', 'request_id': 'req-90df913f-6133-4514-9a87-78df70a61d1d'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:18.759169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] The token cache is being invalidate because project e104fcbffb284649a2162c0b52a3c5db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:07:18.759841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:18.761334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-90df913f-6133-4514-9a87-78df70a61d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:18.764265 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1301/2552] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:07:18 2022] DELETE /identity/v3/projects/e104fcbffb284649a2162c0b52a3c5db => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:18.772070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30668bf9-de8b-4315-9e92-f4557b7cef5f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:18.774104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30668bf9-de8b-4315-9e92-f4557b7cef5f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:18.774924 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30668bf9-de8b-4315-9e92-f4557b7cef5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:18.804259 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-30668bf9-de8b-4315-9e92-f4557b7cef5f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:18.805956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-30668bf9-de8b-4315-9e92-f4557b7cef5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:18.888970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1252/2553] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:18.898110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70143d-0ca9-4c87-945f-04b8946d7630 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:18.898605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70143d-0ca9-4c87-945f-04b8946d7630 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:18.898982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70143d-0ca9-4c87-945f-04b8946d7630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:18.930569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-9d70143d-0ca9-4c87-945f-04b8946d7630 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:18.931897 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9d70143d-0ca9-4c87-945f-04b8946d7630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:18.951040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1302/2554] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:18.952313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30668bf9-de8b-4315-9e92-f4557b7cef5f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:18.972155 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30668bf9-de8b-4315-9e92-f4557b7cef5f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:18.978403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244ba19-8053-48af-bbc1-5817c1393c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:18.979940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244ba19-8053-48af-bbc1-5817c1393c64 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:18.980944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244ba19-8053-48af-bbc1-5817c1393c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:18.981730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244ba19-8053-48af-bbc1-5817c1393c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:18.984143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244ba19-8053-48af-bbc1-5817c1393c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:07:18.984746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244ba19-8053-48af-bbc1-5817c1393c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:18.990133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244ba19-8053-48af-bbc1-5817c1393c64 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:19.076851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1253/2555] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:07:19.080968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70143d-0ca9-4c87-945f-04b8946d7630 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:19.092719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70143d-0ca9-4c87-945f-04b8946d7630 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:19.099508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95b36-aa9c-479c-8ee1-5313fdf8aafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:19.100356 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95b36-aa9c-479c-8ee1-5313fdf8aafb admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:19.100672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95b36-aa9c-479c-8ee1-5313fdf8aafb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:19.101001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95b36-aa9c-479c-8ee1-5313fdf8aafb admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:19.101745 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a95b36-aa9c-479c-8ee1-5313fdf8aafb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:19.103853 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a95b36-aa9c-479c-8ee1-5313fdf8aafb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:19.260559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1303/2556] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:07:19 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:19.265458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244ba19-8053-48af-bbc1-5817c1393c64 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:19.286595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244ba19-8053-48af-bbc1-5817c1393c64 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:19.294131 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:19.471531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06ae96-9ba2-478a-80be-2ddfd27294ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:19.472252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06ae96-9ba2-478a-80be-2ddfd27294ce admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:19.472545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06ae96-9ba2-478a-80be-2ddfd27294ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:19.472866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06ae96-9ba2-478a-80be-2ddfd27294ce admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:19.473700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06ae96-9ba2-478a-80be-2ddfd27294ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:19.475768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06ae96-9ba2-478a-80be-2ddfd27294ce admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:19.621199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1254/2557] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:07:19 2022] POST /identity/v3/users => generated 294 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:19.622718 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95b36-aa9c-479c-8ee1-5313fdf8aafb admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:19.640715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95b36-aa9c-479c-8ee1-5313fdf8aafb admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:19.655621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735824ad-d228-45da-bf99-1aa3d93e61c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:19.656867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735824ad-d228-45da-bf99-1aa3d93e61c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:19.657375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735824ad-d228-45da-bf99-1aa3d93e61c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:19.657924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735824ad-d228-45da-bf99-1aa3d93e61c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:19.659348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735824ad-d228-45da-bf99-1aa3d93e61c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:19.662493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735824ad-d228-45da-bf99-1aa3d93e61c6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:19.689790 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1304/2558] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:07:19.694523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06ae96-9ba2-478a-80be-2ddfd27294ce admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:19.728550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06ae96-9ba2-478a-80be-2ddfd27294ce admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:19.744021 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71da57e-36bc-4419-8dca-6072454996b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:19.746755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71da57e-36bc-4419-8dca-6072454996b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:19.747377 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71da57e-36bc-4419-8dca-6072454996b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:19.748749 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71da57e-36bc-4419-8dca-6072454996b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:19.751194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71da57e-36bc-4419-8dca-6072454996b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:19.756958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71da57e-36bc-4419-8dca-6072454996b7 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:19.789780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1255/2559] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:19.795312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735824ad-d228-45da-bf99-1aa3d93e61c6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:19.814917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735824ad-d228-45da-bf99-1aa3d93e61c6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:19.823459 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234c92c-6667-4007-878c-72f504144460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:19.825623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234c92c-6667-4007-878c-72f504144460 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:19.826370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234c92c-6667-4007-878c-72f504144460 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:19.827189 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234c92c-6667-4007-878c-72f504144460 admin admin] PATH_INFO: `/v3/projects/c704a9a0605e400f8705c79b1f129119/users/5f16172fdf2549f18ea9c330140024b9/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:19.859340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234c92c-6667-4007-878c-72f504144460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c704a9a0605e400f8705c79b1f129119, user_id=5f16172fdf2549f18ea9c330140024b9, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:19.865737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234c92c-6667-4007-878c-72f504144460 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:19.973987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1305/2560] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:19 2022] PUT /identity/v3/projects/c704a9a0605e400f8705c79b1f129119/users/5f16172fdf2549f18ea9c330140024b9/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:19.987744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd84ba-e16a-42f5-97ac-2d7b8d6e4e0e None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:19.990909 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd84ba-e16a-42f5-97ac-2d7b8d6e4e0e None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:19.991415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd84ba-e16a-42f5-97ac-2d7b8d6e4e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:20.025441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d6fd84ba-e16a-42f5-97ac-2d7b8d6e4e0e None None] MFA Rules not processed for user `5f16172fdf2549f18ea9c330140024b9`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:20.028663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d6fd84ba-e16a-42f5-97ac-2d7b8d6e4e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:20.124601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1256/2561] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:20.129641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234c92c-6667-4007-878c-72f504144460 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:20.327423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234c92c-6667-4007-878c-72f504144460 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:20.338137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25589e7-0ac7-4591-86d7-552885292eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:20.342394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25589e7-0ac7-4591-86d7-552885292eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:20.342394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25589e7-0ac7-4591-86d7-552885292eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:20.342394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25589e7-0ac7-4591-86d7-552885292eb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:20.342394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25589e7-0ac7-4591-86d7-552885292eb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:20.344511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25589e7-0ac7-4591-86d7-552885292eb1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:20.466294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1306/2562] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:07:20 2022] POST /identity/v3/projects => generated 389 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:20.467141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd84ba-e16a-42f5-97ac-2d7b8d6e4e0e None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:20.508923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd84ba-e16a-42f5-97ac-2d7b8d6e4e0e None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:20.676480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41220b16-6c7e-43f9-8c28-888bd4465266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:20.677146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41220b16-6c7e-43f9-8c28-888bd4465266 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:20.677445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41220b16-6c7e-43f9-8c28-888bd4465266 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:20.678385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41220b16-6c7e-43f9-8c28-888bd4465266 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:20.680786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41220b16-6c7e-43f9-8c28-888bd4465266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:20.683668 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41220b16-6c7e-43f9-8c28-888bd4465266 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:20.820415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1257/2563] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:07:20 2022] POST /identity/v3/users => generated 301 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:07:20.827142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25589e7-0ac7-4591-86d7-552885292eb1 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:20.877315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25589e7-0ac7-4591-86d7-552885292eb1 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:20.908734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0109ad-055f-4c2d-9a4a-9cc64cd3f302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:20.909888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0109ad-055f-4c2d-9a4a-9cc64cd3f302 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:20.910352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0109ad-055f-4c2d-9a4a-9cc64cd3f302 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:20.910788 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0109ad-055f-4c2d-9a4a-9cc64cd3f302 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:20.912181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0109ad-055f-4c2d-9a4a-9cc64cd3f302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:20.915841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0109ad-055f-4c2d-9a4a-9cc64cd3f302 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:20.959027 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1307/2564] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:20.973488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41220b16-6c7e-43f9-8c28-888bd4465266 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:20.999683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:20.999683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41220b16-6c7e-43f9-8c28-888bd4465266 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:21.021205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24643d0-e2c9-456e-9be9-f7b37630c537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:21.021205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24643d0-e2c9-456e-9be9-f7b37630c537 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:21.021733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24643d0-e2c9-456e-9be9-f7b37630c537 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:21.021893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24643d0-e2c9-456e-9be9-f7b37630c537 admin admin] PATH_INFO: `/v3/projects/abbc5ee3b50145b4acec6ef28902e5f0/users/397b2e20838b43f9a0645e8349ce106b/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:21.055441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24643d0-e2c9-456e-9be9-f7b37630c537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abbc5ee3b50145b4acec6ef28902e5f0, user_id=397b2e20838b43f9a0645e8349ce106b, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:21.059207 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24643d0-e2c9-456e-9be9-f7b37630c537 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:21.266950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1258/2565] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:20 2022] PUT /identity/v3/projects/abbc5ee3b50145b4acec6ef28902e5f0/users/397b2e20838b43f9a0645e8349ce106b/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:21.272244 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0109ad-055f-4c2d-9a4a-9cc64cd3f302 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:21.298781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0109ad-055f-4c2d-9a4a-9cc64cd3f302 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:21.470815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:21.568700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1e9ed-f7cd-468d-af12-32167a376729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:21.569613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1e9ed-f7cd-468d-af12-32167a376729 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:21.577325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1e9ed-f7cd-468d-af12-32167a376729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:21.577325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1e9ed-f7cd-468d-af12-32167a376729 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:21.579829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1e9ed-f7cd-468d-af12-32167a376729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:21.590341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1e9ed-f7cd-468d-af12-32167a376729 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:21.628868 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1308/2566] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:07:21.635426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24643d0-e2c9-456e-9be9-f7b37630c537 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:21.652556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24643d0-e2c9-456e-9be9-f7b37630c537 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:21.662709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9c34c-2c12-4a16-83c7-ff7ccc1f8fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:21.662709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9c34c-2c12-4a16-83c7-ff7ccc1f8fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:21.662709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9c34c-2c12-4a16-83c7-ff7ccc1f8fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:21.662709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9c34c-2c12-4a16-83c7-ff7ccc1f8fa2 admin admin] PATH_INFO: `/v3/projects/abbc5ee3b50145b4acec6ef28902e5f0/users/397b2e20838b43f9a0645e8349ce106b/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:21.663339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9c34c-2c12-4a16-83c7-ff7ccc1f8fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abbc5ee3b50145b4acec6ef28902e5f0, user_id=397b2e20838b43f9a0645e8349ce106b, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:21.667161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9c34c-2c12-4a16-83c7-ff7ccc1f8fa2 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:21.686033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1259/2567] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:21 2022] PUT /identity/v3/projects/abbc5ee3b50145b4acec6ef28902e5f0/users/397b2e20838b43f9a0645e8349ce106b/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:21.713183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb950025-25d8-4e7c-9226-97b10c1414c1 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:21.714480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb950025-25d8-4e7c-9226-97b10c1414c1 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:21.715193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb950025-25d8-4e7c-9226-97b10c1414c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:21.786642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-bb950025-25d8-4e7c-9226-97b10c1414c1 None None] MFA Rules not processed for user `397b2e20838b43f9a0645e8349ce106b`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:21.789563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-bb950025-25d8-4e7c-9226-97b10c1414c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:22.022832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1309/2568] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:22.039249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9c34c-2c12-4a16-83c7-ff7ccc1f8fa2 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:22.041743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2fc9c34c-2c12-4a16-83c7-ff7ccc1f8fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:22.146275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9c34c-2c12-4a16-83c7-ff7ccc1f8fa2 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:22.152578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317d308-556a-46b9-964f-b06b4dd8b7e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:22.153681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317d308-556a-46b9-964f-b06b4dd8b7e6 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:22.154213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317d308-556a-46b9-964f-b06b4dd8b7e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:22.154872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317d308-556a-46b9-964f-b06b4dd8b7e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:22.174515 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317d308-556a-46b9-964f-b06b4dd8b7e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:22.176609 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317d308-556a-46b9-964f-b06b4dd8b7e6 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:22.194105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1260/2569] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:22.668685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb950025-25d8-4e7c-9226-97b10c1414c1 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:22.949614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb950025-25d8-4e7c-9226-97b10c1414c1 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:22.953215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:22.955879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:22.956351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:22.956815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] PATH_INFO: `/v3/users/5f16172fdf2549f18ea9c330140024b9` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:22.958991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f16172fdf2549f18ea9c330140024b9)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:22.962655 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:23.136474 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:23.439363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f16172fdf2549f18ea9c330140024b9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:23.453876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f16172fdf2549f18ea9c330140024b9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:23.465192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f16172fdf2549f18ea9c330140024b9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:23.466269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f16172fdf2549f18ea9c330140024b9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:23.566633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f16172fdf2549f18ea9c330140024b9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:23.582886 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1310/2570] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:07:22 2022] DELETE /identity/v3/users/5f16172fdf2549f18ea9c330140024b9 => generated 0 bytes in 916 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:23.591446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7648cbb-002f-4cdd-970c-45c02291960b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:23.592907 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7648cbb-002f-4cdd-970c-45c02291960b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:23.593672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7648cbb-002f-4cdd-970c-45c02291960b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:23.707837 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d7648cbb-002f-4cdd-970c-45c02291960b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:23.709408 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d7648cbb-002f-4cdd-970c-45c02291960b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:23.794988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1261/2571] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:23.806727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:23.809219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:23.951504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfe096-8126-4044-b041-5f3856ffc827 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:23.951802 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfe096-8126-4044-b041-5f3856ffc827 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:23.952308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfe096-8126-4044-b041-5f3856ffc827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:23.964693 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd66150-6d09-4f82-b913-8de93c296012 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:23.972408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29c852-b4cb-4d47-8be8-d9b11de826a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:23.973612 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29c852-b4cb-4d47-8be8-d9b11de826a2 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:23.974127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29c852-b4cb-4d47-8be8-d9b11de826a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:23.974633 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29c852-b4cb-4d47-8be8-d9b11de826a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:23.976790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-9ebfe096-8126-4044-b041-5f3856ffc827 None None] MFA Rules not processed for user `7ed43701d36b4b479a2a3a978d242625`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:23.979052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9ebfe096-8126-4044-b041-5f3856ffc827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:23.992865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29c852-b4cb-4d47-8be8-d9b11de826a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:23.994923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29c852-b4cb-4d47-8be8-d9b11de826a2 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:24.007766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1311/2572] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:07:24.093255 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:24.112546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1262/2573] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:24.649780 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29c852-b4cb-4d47-8be8-d9b11de826a2 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:24.652325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ad29c852-b4cb-4d47-8be8-d9b11de826a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:24.731109 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfe096-8126-4044-b041-5f3856ffc827 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:24.733816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9ebfe096-8126-4044-b041-5f3856ffc827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:24.740854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29c852-b4cb-4d47-8be8-d9b11de826a2 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:24.754492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db734519-ee59-4385-9d3b-361d7000d457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:24.756160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db734519-ee59-4385-9d3b-361d7000d457 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:24.757290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db734519-ee59-4385-9d3b-361d7000d457 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:24.759734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db734519-ee59-4385-9d3b-361d7000d457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:24.762894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-db734519-ee59-4385-9d3b-361d7000d457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:24.765737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfe096-8126-4044-b041-5f3856ffc827 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:24.776366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:24.776898 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db734519-ee59-4385-9d3b-361d7000d457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:24.778027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:24.778704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:24.779239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] PATH_INFO: `/v3/projects/c704a9a0605e400f8705c79b1f129119` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:24.779877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db734519-ee59-4385-9d3b-361d7000d457 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:24.781796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c704a9a0605e400f8705c79b1f129119)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:24.786116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:24.793274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1312/2574] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:24.934957 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c704a9a0605e400f8705c79b1f129119', 'request_id': 'req-c6abed32-bf19-43e0-81ae-89d458811b16'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:24.942545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c704a9a0605e400f8705c79b1f129119', 'request_id': 'req-c6abed32-bf19-43e0-81ae-89d458811b16'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:25.027536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c704a9a0605e400f8705c79b1f129119', 'request_id': 'req-c6abed32-bf19-43e0-81ae-89d458811b16'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:25.044981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] The token cache is being invalidate because project c704a9a0605e400f8705c79b1f129119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:07:25.045321 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:25.050942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:25.050942 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1263/2575] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:07:24 2022] DELETE /identity/v3/projects/c704a9a0605e400f8705c79b1f129119 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:25.052353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db734519-ee59-4385-9d3b-361d7000d457 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:25.072118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db734519-ee59-4385-9d3b-361d7000d457 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:25.155488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:25.156420 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:25.156749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:25.157515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] PATH_INFO: `/v3/users/397b2e20838b43f9a0645e8349ce106b` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:25.158342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=397b2e20838b43f9a0645e8349ce106b)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:25.160361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:25.162217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:25.436858 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '397b2e20838b43f9a0645e8349ce106b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:25.456412 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '397b2e20838b43f9a0645e8349ce106b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:25.465191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '397b2e20838b43f9a0645e8349ce106b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:25.466095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '397b2e20838b43f9a0645e8349ce106b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:25.560720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-3f16a715-9d75-4436-859e-dcf49c2d8084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '397b2e20838b43f9a0645e8349ce106b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:25.568206 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1313/2576] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:07:25 2022] DELETE /identity/v3/users/397b2e20838b43f9a0645e8349ce106b => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:07:26.221996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:26.221996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:26.262250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abed32-bf19-43e0-81ae-89d458811b16 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:26.338007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:26.338946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:26.339420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:26.339713 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] PATH_INFO: `/v3/projects/abbc5ee3b50145b4acec6ef28902e5f0` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:26.341067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abbc5ee3b50145b4acec6ef28902e5f0)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:26.343269 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:26.535687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abbc5ee3b50145b4acec6ef28902e5f0', 'request_id': 'req-52cf1242-3490-4ec1-bc86-0cf630494c66'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:26.546540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abbc5ee3b50145b4acec6ef28902e5f0', 'request_id': 'req-52cf1242-3490-4ec1-bc86-0cf630494c66'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:26.644614 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abbc5ee3b50145b4acec6ef28902e5f0', 'request_id': 'req-52cf1242-3490-4ec1-bc86-0cf630494c66'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:26.660672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] The token cache is being invalidate because project abbc5ee3b50145b4acec6ef28902e5f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:07:26.661117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:26.661968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-52cf1242-3490-4ec1-bc86-0cf630494c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:26.663939 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1264/2577] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:07:26 2022] DELETE /identity/v3/projects/abbc5ee3b50145b4acec6ef28902e5f0 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:26.670387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4cb54-5866-4893-9efa-675326b9d33a None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:26.670973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4cb54-5866-4893-9efa-675326b9d33a None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:26.671430 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4cb54-5866-4893-9efa-675326b9d33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:26.749758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-dae4cb54-5866-4893-9efa-675326b9d33a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:26.751811 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-dae4cb54-5866-4893-9efa-675326b9d33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:26.873959 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1314/2578] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:07:26.887842 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92117579-5fdc-440a-a375-ca9318ced828 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:26.888329 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92117579-5fdc-440a-a375-ca9318ced828 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:26.889519 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92117579-5fdc-440a-a375-ca9318ced828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:26.904359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:26.927950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-92117579-5fdc-440a-a375-ca9318ced828 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:26.930499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-92117579-5fdc-440a-a375-ca9318ced828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:26.955591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1265/2579] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:26.955966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4cb54-5866-4893-9efa-675326b9d33a None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:26.972331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4cb54-5866-4893-9efa-675326b9d33a None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:26.985360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfe251-f3b7-4254-8056-f2678d1abe48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:26.986078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfe251-f3b7-4254-8056-f2678d1abe48 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:26.986373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfe251-f3b7-4254-8056-f2678d1abe48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:26.986656 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfe251-f3b7-4254-8056-f2678d1abe48 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:26.987577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfe251-f3b7-4254-8056-f2678d1abe48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:07:26.987891 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfe251-f3b7-4254-8056-f2678d1abe48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:26.989588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfe251-f3b7-4254-8056-f2678d1abe48 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:27.009728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1315/2580] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:07:27.012138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92117579-5fdc-440a-a375-ca9318ced828 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:27.036447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92117579-5fdc-440a-a375-ca9318ced828 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:27.045492 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf3fcf-736e-41b2-90f7-506d417783a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:27.046785 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf3fcf-736e-41b2-90f7-506d417783a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:27.047368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf3fcf-736e-41b2-90f7-506d417783a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:27.047967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf3fcf-736e-41b2-90f7-506d417783a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:27.050280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf3fcf-736e-41b2-90f7-506d417783a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:27.055444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf3fcf-736e-41b2-90f7-506d417783a7 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:27.226285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1266/2581] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:07:27 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:27.231937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfe251-f3b7-4254-8056-f2678d1abe48 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:27.249046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfe251-f3b7-4254-8056-f2678d1abe48 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:27.314882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:27.335912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f004928-4b35-463e-9b72-ffb45cbdb28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:27.336620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f004928-4b35-463e-9b72-ffb45cbdb28d admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:27.336941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f004928-4b35-463e-9b72-ffb45cbdb28d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:27.337224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f004928-4b35-463e-9b72-ffb45cbdb28d admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:27.337943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f004928-4b35-463e-9b72-ffb45cbdb28d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:27.340835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f004928-4b35-463e-9b72-ffb45cbdb28d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:27.456761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1316/2582] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:07:27 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:27.461562 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf3fcf-736e-41b2-90f7-506d417783a7 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:27.475431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf3fcf-736e-41b2-90f7-506d417783a7 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:27.481679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b45188-b13a-432a-957c-aac13ef2c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:27.482763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b45188-b13a-432a-957c-aac13ef2c7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:27.483230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b45188-b13a-432a-957c-aac13ef2c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:27.483756 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b45188-b13a-432a-957c-aac13ef2c7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:27.485090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b45188-b13a-432a-957c-aac13ef2c7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:27.487046 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b45188-b13a-432a-957c-aac13ef2c7d7 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:27.499982 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1267/2583] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:27.505059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f004928-4b35-463e-9b72-ffb45cbdb28d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:27.518437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f004928-4b35-463e-9b72-ffb45cbdb28d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:27.523842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad38760-8d0f-42a5-8f85-a5b7c5fc9458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:27.524582 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad38760-8d0f-42a5-8f85-a5b7c5fc9458 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:27.524911 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad38760-8d0f-42a5-8f85-a5b7c5fc9458 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:27.525230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad38760-8d0f-42a5-8f85-a5b7c5fc9458 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:27.526092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad38760-8d0f-42a5-8f85-a5b7c5fc9458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:27.528391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad38760-8d0f-42a5-8f85-a5b7c5fc9458 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:27.541624 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1317/2584] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:27.545397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b45188-b13a-432a-957c-aac13ef2c7d7 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:27.560571 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b45188-b13a-432a-957c-aac13ef2c7d7 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:27.568194 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d23718-db9b-48d5-b653-8eaece28d41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:27.569687 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d23718-db9b-48d5-b653-8eaece28d41e admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:27.570177 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d23718-db9b-48d5-b653-8eaece28d41e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:27.570707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d23718-db9b-48d5-b653-8eaece28d41e admin admin] PATH_INFO: `/v3/projects/583b755ee44648ecbe1284a7c153a17d/users/4478a16bf8b849e8a36794ae144ad5fc/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:27.600438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d23718-db9b-48d5-b653-8eaece28d41e admin admin] RBAC: Authorizing `identity:create_grant(project_id=583b755ee44648ecbe1284a7c153a17d, user_id=4478a16bf8b849e8a36794ae144ad5fc, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:27.604418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d23718-db9b-48d5-b653-8eaece28d41e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:27.682054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1268/2585] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:27 2022] PUT /identity/v3/projects/583b755ee44648ecbe1284a7c153a17d/users/4478a16bf8b849e8a36794ae144ad5fc/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:27.699335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419f820-9817-4fbe-b468-30b78e295719 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:27.699924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419f820-9817-4fbe-b468-30b78e295719 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:27.700221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419f820-9817-4fbe-b468-30b78e295719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:27.742708 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-8419f820-9817-4fbe-b468-30b78e295719 None None] MFA Rules not processed for user `4478a16bf8b849e8a36794ae144ad5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:27.744173 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8419f820-9817-4fbe-b468-30b78e295719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:27.853146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1318/2586] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:27.859038 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d23718-db9b-48d5-b653-8eaece28d41e admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:27.963553 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d23718-db9b-48d5-b653-8eaece28d41e admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:27.970618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f5cfd-eb26-4e12-9489-406614015e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:27.971632 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f5cfd-eb26-4e12-9489-406614015e0b admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:27.972062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f5cfd-eb26-4e12-9489-406614015e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:27.972491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f5cfd-eb26-4e12-9489-406614015e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:27.974017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f5cfd-eb26-4e12-9489-406614015e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:27.977980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f5cfd-eb26-4e12-9489-406614015e0b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:28.154545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1269/2587] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:07:27 2022] POST /identity/v3/projects => generated 387 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:28.159183 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419f820-9817-4fbe-b468-30b78e295719 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:28.172578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419f820-9817-4fbe-b468-30b78e295719 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:28.273947 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171022a-8521-451e-b923-34a119b1438d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:28.274966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171022a-8521-451e-b923-34a119b1438d admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:28.275322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171022a-8521-451e-b923-34a119b1438d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:28.275526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171022a-8521-451e-b923-34a119b1438d admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:28.276285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171022a-8521-451e-b923-34a119b1438d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:28.278956 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171022a-8521-451e-b923-34a119b1438d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:28.416060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1319/2588] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:07:28 2022] POST /identity/v3/users => generated 300 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:28.417447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f5cfd-eb26-4e12-9489-406614015e0b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:28.429321 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f5cfd-eb26-4e12-9489-406614015e0b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:28.434627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3502da0-426a-407a-9f62-42de5810b7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:28.435313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3502da0-426a-407a-9f62-42de5810b7ea admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:28.435604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3502da0-426a-407a-9f62-42de5810b7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:28.435885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3502da0-426a-407a-9f62-42de5810b7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:28.436622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3502da0-426a-407a-9f62-42de5810b7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:28.438249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3502da0-426a-407a-9f62-42de5810b7ea admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:28.452887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1270/2589] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:28.457230 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171022a-8521-451e-b923-34a119b1438d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:28.479822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171022a-8521-451e-b923-34a119b1438d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:28.484909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fb0f8-3fc0-4fd6-aa2a-d6e2561b777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:28.485859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fb0f8-3fc0-4fd6-aa2a-d6e2561b777f admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:28.486145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fb0f8-3fc0-4fd6-aa2a-d6e2561b777f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:28.486422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fb0f8-3fc0-4fd6-aa2a-d6e2561b777f admin admin] PATH_INFO: `/v3/projects/0a5b8e79c05c4a90b88bf14ffc7e2f49/users/7b32b270bfa04b4db99726b62a3bc52a/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:28.503074 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9fb0f8-3fc0-4fd6-aa2a-d6e2561b777f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5b8e79c05c4a90b88bf14ffc7e2f49, user_id=7b32b270bfa04b4db99726b62a3bc52a, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:28.506041 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9fb0f8-3fc0-4fd6-aa2a-d6e2561b777f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:28.634342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1320/2590] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:28 2022] PUT /identity/v3/projects/0a5b8e79c05c4a90b88bf14ffc7e2f49/users/7b32b270bfa04b4db99726b62a3bc52a/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:28.635557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3502da0-426a-407a-9f62-42de5810b7ea admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:28.651540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3502da0-426a-407a-9f62-42de5810b7ea admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:28.821419 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f251bf3-dafb-4547-be2a-8abc5319d53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:28.822954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f251bf3-dafb-4547-be2a-8abc5319d53a admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:28.823691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f251bf3-dafb-4547-be2a-8abc5319d53a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:28.824337 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f251bf3-dafb-4547-be2a-8abc5319d53a admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:28.825974 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f251bf3-dafb-4547-be2a-8abc5319d53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:28.829459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f251bf3-dafb-4547-be2a-8abc5319d53a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:28.841285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:28.858081 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fb0f8-3fc0-4fd6-aa2a-d6e2561b777f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:28.860846 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1271/2591] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:28.880216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fb0f8-3fc0-4fd6-aa2a-d6e2561b777f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:28.888161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f30744-6b27-408d-b4c3-428dcc24261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:28.893341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f30744-6b27-408d-b4c3-428dcc24261f admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:28.893341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f30744-6b27-408d-b4c3-428dcc24261f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:28.893341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f30744-6b27-408d-b4c3-428dcc24261f admin admin] PATH_INFO: `/v3/projects/0a5b8e79c05c4a90b88bf14ffc7e2f49/users/7b32b270bfa04b4db99726b62a3bc52a/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:28.893341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f30744-6b27-408d-b4c3-428dcc24261f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5b8e79c05c4a90b88bf14ffc7e2f49, user_id=7b32b270bfa04b4db99726b62a3bc52a, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:28.895709 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f30744-6b27-408d-b4c3-428dcc24261f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:28.920118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1321/2592] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:28 2022] PUT /identity/v3/projects/0a5b8e79c05c4a90b88bf14ffc7e2f49/users/7b32b270bfa04b4db99726b62a3bc52a/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:28.944850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeb6ef-9a15-4b0c-8c81-1696af6dd953 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:28.945153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeb6ef-9a15-4b0c-8c81-1696af6dd953 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:28.945444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeb6ef-9a15-4b0c-8c81-1696af6dd953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:28.976382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-27aeb6ef-9a15-4b0c-8c81-1696af6dd953 None None] MFA Rules not processed for user `7b32b270bfa04b4db99726b62a3bc52a`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:28.977991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-27aeb6ef-9a15-4b0c-8c81-1696af6dd953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:29.098813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1272/2593] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:07:29.109948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f30744-6b27-408d-b4c3-428dcc24261f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:29.112067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e8f30744-6b27-408d-b4c3-428dcc24261f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:29.116233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:29.224615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f30744-6b27-408d-b4c3-428dcc24261f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:29.229287 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62f232-5ad1-4f3a-8116-768d2909f62b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:29.231941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62f232-5ad1-4f3a-8116-768d2909f62b service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:29.231941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62f232-5ad1-4f3a-8116-768d2909f62b service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:29.231941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62f232-5ad1-4f3a-8116-768d2909f62b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:29.248496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62f232-5ad1-4f3a-8116-768d2909f62b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:29.251819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62f232-5ad1-4f3a-8116-768d2909f62b service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:29.276164 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1322/2594] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:07:31.022804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeb6ef-9a15-4b0c-8c81-1696af6dd953 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:31.108313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeb6ef-9a15-4b0c-8c81-1696af6dd953 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:31.122806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:31.123840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:31.124322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:31.124970 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] PATH_INFO: `/v3/users/4478a16bf8b849e8a36794ae144ad5fc` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:31.126339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4478a16bf8b849e8a36794ae144ad5fc)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:31.128367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:31.394176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4478a16bf8b849e8a36794ae144ad5fc'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:31.404900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4478a16bf8b849e8a36794ae144ad5fc'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:31.405671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4478a16bf8b849e8a36794ae144ad5fc'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:31.515270 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4478a16bf8b849e8a36794ae144ad5fc'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:31.548037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4478a16bf8b849e8a36794ae144ad5fc'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:31.556327 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1273/2595] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:07:31 2022] DELETE /identity/v3/users/4478a16bf8b849e8a36794ae144ad5fc => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:31.561296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175ff6d-210b-4c24-bfbb-09f14580a326 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:31.561749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175ff6d-210b-4c24-bfbb-09f14580a326 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:31.561997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175ff6d-210b-4c24-bfbb-09f14580a326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:31.586573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a175ff6d-210b-4c24-bfbb-09f14580a326 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:31.586573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a175ff6d-210b-4c24-bfbb-09f14580a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:31.743335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1323/2596] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:31.762205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:31.763740 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:31.773091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:31.878384 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7662a5a-2817-43b7-b9dd-c242a5e0b13d admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:31.882934 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54648df-e046-48fa-9698-31b8795ecda4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:31.883782 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54648df-e046-48fa-9698-31b8795ecda4 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:31.884730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54648df-e046-48fa-9698-31b8795ecda4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:31.888808 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54648df-e046-48fa-9698-31b8795ecda4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:31.904407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54648df-e046-48fa-9698-31b8795ecda4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:31.906249 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54648df-e046-48fa-9698-31b8795ecda4 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:31.918398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1274/2597] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:07:32.803742 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175ff6d-210b-4c24-bfbb-09f14580a326 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:32.806187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a175ff6d-210b-4c24-bfbb-09f14580a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:32.846094 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175ff6d-210b-4c24-bfbb-09f14580a326 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:32.859814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:32.861399 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:32.861934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:32.862445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] PATH_INFO: `/v3/projects/583b755ee44648ecbe1284a7c153a17d` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:32.865129 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=583b755ee44648ecbe1284a7c153a17d)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:32.869178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:33.064750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583b755ee44648ecbe1284a7c153a17d', 'request_id': 'req-7bf304ba-4bcc-4a74-a75e-df561ab5b737'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.076445 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583b755ee44648ecbe1284a7c153a17d', 'request_id': 'req-7bf304ba-4bcc-4a74-a75e-df561ab5b737'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.095392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583b755ee44648ecbe1284a7c153a17d', 'request_id': 'req-7bf304ba-4bcc-4a74-a75e-df561ab5b737'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.208315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] The token cache is being invalidate because project 583b755ee44648ecbe1284a7c153a17d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:07:33.208885 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.217471 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.220930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1324/2598] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:07:32 2022] DELETE /identity/v3/projects/583b755ee44648ecbe1284a7c153a17d => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:33.224707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54648df-e046-48fa-9698-31b8795ecda4 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:33.239730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54648df-e046-48fa-9698-31b8795ecda4 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:33.376319 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:33.381257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:33.381257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:33.381257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] PATH_INFO: `/v3/users/7b32b270bfa04b4db99726b62a3bc52a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:33.381257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b32b270bfa04b4db99726b62a3bc52a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:33.389439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:33.665575 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b32b270bfa04b4db99726b62a3bc52a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.695884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b32b270bfa04b4db99726b62a3bc52a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.696894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b32b270bfa04b4db99726b62a3bc52a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.829228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b32b270bfa04b4db99726b62a3bc52a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.861447 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-00231e3d-8e1c-4773-8dbf-ca64ce3bc2ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b32b270bfa04b4db99726b62a3bc52a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:33.874971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1275/2599] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:07:33 2022] DELETE /identity/v3/users/7b32b270bfa04b4db99726b62a3bc52a => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:34.462466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:34.464603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:34.513675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf304ba-4bcc-4a74-a75e-df561ab5b737 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:34.521690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:34.595335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:34.596274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:34.596534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:34.596864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] PATH_INFO: `/v3/projects/0a5b8e79c05c4a90b88bf14ffc7e2f49` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:34.598103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a5b8e79c05c4a90b88bf14ffc7e2f49)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:34.600444 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:34.742997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a5b8e79c05c4a90b88bf14ffc7e2f49', 'request_id': 'req-98d6ab02-d34d-494b-9e16-92e42419f0eb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:34.751549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a5b8e79c05c4a90b88bf14ffc7e2f49', 'request_id': 'req-98d6ab02-d34d-494b-9e16-92e42419f0eb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:34.760194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a5b8e79c05c4a90b88bf14ffc7e2f49', 'request_id': 'req-98d6ab02-d34d-494b-9e16-92e42419f0eb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:34.870544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] The token cache is being invalidate because project 0a5b8e79c05c4a90b88bf14ffc7e2f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:07:34.870859 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:34.871712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-98d6ab02-d34d-494b-9e16-92e42419f0eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:34.873336 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1325/2600] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:07:34 2022] DELETE /identity/v3/projects/0a5b8e79c05c4a90b88bf14ffc7e2f49 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:34.878647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc71464-f741-4ae8-8bdd-bfbed834cf26 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:34.879480 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc71464-f741-4ae8-8bdd-bfbed834cf26 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:34.879671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc71464-f741-4ae8-8bdd-bfbed834cf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:34.911171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-bfc71464-f741-4ae8-8bdd-bfbed834cf26 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:34.913028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-bfc71464-f741-4ae8-8bdd-bfbed834cf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:34.955490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:35.017671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1276/2601] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:35.019613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbae835-1745-4d05-a675-cf2381d40a8d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:35.020148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbae835-1745-4d05-a675-cf2381d40a8d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:35.020450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbae835-1745-4d05-a675-cf2381d40a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:35.042700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-6cbae835-1745-4d05-a675-cf2381d40a8d None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:35.043957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6cbae835-1745-4d05-a675-cf2381d40a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:35.052545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1326/2602] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:07:35.057076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc71464-f741-4ae8-8bdd-bfbed834cf26 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:35.069230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc71464-f741-4ae8-8bdd-bfbed834cf26 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:35.075869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f40b-6341-4d80-bcc7-9891c4e22f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:35.077174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f40b-6341-4d80-bcc7-9891c4e22f3d admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:35.077695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f40b-6341-4d80-bcc7-9891c4e22f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:35.078183 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f40b-6341-4d80-bcc7-9891c4e22f3d admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:35.079691 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106f40b-6341-4d80-bcc7-9891c4e22f3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:07:35.080224 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106f40b-6341-4d80-bcc7-9891c4e22f3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:35.082989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106f40b-6341-4d80-bcc7-9891c4e22f3d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:35.107448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1277/2603] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:07:35.113438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbae835-1745-4d05-a675-cf2381d40a8d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:35.137520 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbae835-1745-4d05-a675-cf2381d40a8d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:35.151078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f7605-90d2-4e96-9faf-6cb1019fecaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:35.152275 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7605-90d2-4e96-9faf-6cb1019fecaf admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:35.152822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7605-90d2-4e96-9faf-6cb1019fecaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:35.154389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7605-90d2-4e96-9faf-6cb1019fecaf admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:35.159593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7605-90d2-4e96-9faf-6cb1019fecaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:35.162074 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7605-90d2-4e96-9faf-6cb1019fecaf admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:35.286657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1327/2604] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:07:35 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:35.291236 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f40b-6341-4d80-bcc7-9891c4e22f3d admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:35.302469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f40b-6341-4d80-bcc7-9891c4e22f3d admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:35.368978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99303557-ed05-4e66-988c-0816637a5b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:35.369581 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303557-ed05-4e66-988c-0816637a5b84 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:35.369869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303557-ed05-4e66-988c-0816637a5b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:35.370212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303557-ed05-4e66-988c-0816637a5b84 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:35.371451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99303557-ed05-4e66-988c-0816637a5b84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:35.373401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99303557-ed05-4e66-988c-0816637a5b84 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:35.461605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1278/2605] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:07:35 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:35.463053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f7605-90d2-4e96-9faf-6cb1019fecaf admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:35.481783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f7605-90d2-4e96-9faf-6cb1019fecaf admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:35.490139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60da15-4a16-4e29-be0f-f453b3b31370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:35.490839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60da15-4a16-4e29-be0f-f453b3b31370 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:35.491126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60da15-4a16-4e29-be0f-f453b3b31370 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:35.491402 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60da15-4a16-4e29-be0f-f453b3b31370 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:35.492142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60da15-4a16-4e29-be0f-f453b3b31370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:35.494496 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60da15-4a16-4e29-be0f-f453b3b31370 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:35.506229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1328/2606] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:07:35.510181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99303557-ed05-4e66-988c-0816637a5b84 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:35.521832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99303557-ed05-4e66-988c-0816637a5b84 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:35.530477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba9b20-5fa5-458c-8f3f-7e5e5d98f4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:35.531582 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba9b20-5fa5-458c-8f3f-7e5e5d98f4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:35.532047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba9b20-5fa5-458c-8f3f-7e5e5d98f4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:35.532491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba9b20-5fa5-458c-8f3f-7e5e5d98f4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:35.533513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba9b20-5fa5-458c-8f3f-7e5e5d98f4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:35.535182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba9b20-5fa5-458c-8f3f-7e5e5d98f4b7 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:35.559448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1279/2607] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:35.563727 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60da15-4a16-4e29-be0f-f453b3b31370 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:35.575516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60da15-4a16-4e29-be0f-f453b3b31370 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:35.581141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164fd53-2773-4163-9b1b-caa220e8f676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:35.582567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164fd53-2773-4163-9b1b-caa220e8f676 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:35.583160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164fd53-2773-4163-9b1b-caa220e8f676 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:35.583694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164fd53-2773-4163-9b1b-caa220e8f676 admin admin] PATH_INFO: `/v3/projects/3c641cc60c3c4e21add812894e369f35/users/95526423fa9744b592b744c9f4e039c2/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:35.603551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164fd53-2773-4163-9b1b-caa220e8f676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c641cc60c3c4e21add812894e369f35, user_id=95526423fa9744b592b744c9f4e039c2, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:35.607122 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164fd53-2773-4163-9b1b-caa220e8f676 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:35.715972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1329/2608] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:35 2022] PUT /identity/v3/projects/3c641cc60c3c4e21add812894e369f35/users/95526423fa9744b592b744c9f4e039c2/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:35.727310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb05f1-a7a8-4c3e-8cbd-3cb14ae99ec3 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:35.727622 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb05f1-a7a8-4c3e-8cbd-3cb14ae99ec3 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:35.727892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb05f1-a7a8-4c3e-8cbd-3cb14ae99ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:35.761947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-47fb05f1-a7a8-4c3e-8cbd-3cb14ae99ec3 None None] MFA Rules not processed for user `95526423fa9744b592b744c9f4e039c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:35.763256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-47fb05f1-a7a8-4c3e-8cbd-3cb14ae99ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:35.856816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1280/2609] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:35.869454 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164fd53-2773-4163-9b1b-caa220e8f676 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:35.871542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f164fd53-2773-4163-9b1b-caa220e8f676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:35.931451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:35.952774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164fd53-2773-4163-9b1b-caa220e8f676 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:35.956585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984d007-972c-4443-8023-8acc5528748e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:35.957446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984d007-972c-4443-8023-8acc5528748e service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:35.957800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984d007-972c-4443-8023-8acc5528748e service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:35.958162 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984d007-972c-4443-8023-8acc5528748e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:35.967819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984d007-972c-4443-8023-8acc5528748e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:35.969486 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984d007-972c-4443-8023-8acc5528748e service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:35.982712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1330/2610] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:36.305323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91614e81-e8d6-47aa-96be-2d0f052efbef None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:36.306231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91614e81-e8d6-47aa-96be-2d0f052efbef None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:36.306685 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91614e81-e8d6-47aa-96be-2d0f052efbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:36.326879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-91614e81-e8d6-47aa-96be-2d0f052efbef None None] MFA Rules not processed for user `822e56f086f0481ca4c6970adf85e396`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:36.328410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-91614e81-e8d6-47aa-96be-2d0f052efbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:36.348776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:36.390600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1281/2611] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:36.622239 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1178f-1bea-497e-8cce-e16a122ebd80 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:36.623132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1178f-1bea-497e-8cce-e16a122ebd80 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:36.623302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1178f-1bea-497e-8cce-e16a122ebd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:36.667685 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-14f1178f-1bea-497e-8cce-e16a122ebd80 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:36.669374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-14f1178f-1bea-497e-8cce-e16a122ebd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:36.752470 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1331/2612] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:07:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:36.771860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91614e81-e8d6-47aa-96be-2d0f052efbef None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:36.774311 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-91614e81-e8d6-47aa-96be-2d0f052efbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:36.941641 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91614e81-e8d6-47aa-96be-2d0f052efbef None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:36.947258 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a45ffa-c365-4f5d-b342-1a265570fdb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:36.947994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45ffa-c365-4f5d-b342-1a265570fdb3 service swift] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:36.948318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45ffa-c365-4f5d-b342-1a265570fdb3 service swift] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:36.948599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45ffa-c365-4f5d-b342-1a265570fdb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:36.960611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a45ffa-c365-4f5d-b342-1a265570fdb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:36.962751 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a45ffa-c365-4f5d-b342-1a265570fdb3 service swift] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:36.990488 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1282/2613] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:07:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:07:41.148022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1178f-1bea-497e-8cce-e16a122ebd80 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:41.282068 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1178f-1bea-497e-8cce-e16a122ebd80 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:41.289595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:41.291061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:41.291545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:41.292576 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] PATH_INFO: `/v3/users/95526423fa9744b592b744c9f4e039c2` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:41.296014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] RBAC: Authorizing `identity:delete_user(user_id=95526423fa9744b592b744c9f4e039c2)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:41.298654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:41.549461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95526423fa9744b592b744c9f4e039c2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:41.575393 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95526423fa9744b592b744c9f4e039c2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:41.590662 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95526423fa9744b592b744c9f4e039c2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:41.591461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95526423fa9744b592b744c9f4e039c2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:41.595829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:41.664710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95526423fa9744b592b744c9f4e039c2'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:41.672482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1332/2614] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:07:41 2022] DELETE /identity/v3/users/95526423fa9744b592b744c9f4e039c2 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:41.677302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48aecb-02ec-4a09-b4c5-55fe746f280c None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:41.677834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48aecb-02ec-4a09-b4c5-55fe746f280c None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:41.678086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48aecb-02ec-4a09-b4c5-55fe746f280c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:41.705505 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-2c48aecb-02ec-4a09-b4c5-55fe746f280c None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:41.706892 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2c48aecb-02ec-4a09-b4c5-55fe746f280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:41.783655 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1283/2615] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:41.802769 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:41.805132 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:41.904561 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceebd7a-78a3-40fb-bec8-6009a52fd61a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:41.908761 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3ca82-465c-40c8-9ddc-caf68e7fa06e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:41.909454 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3ca82-465c-40c8-9ddc-caf68e7fa06e service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:41.909745 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3ca82-465c-40c8-9ddc-caf68e7fa06e service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:41.910022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3ca82-465c-40c8-9ddc-caf68e7fa06e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:41.921118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3ca82-465c-40c8-9ddc-caf68e7fa06e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:41.922542 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3ca82-465c-40c8-9ddc-caf68e7fa06e service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:41.934534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1333/2616] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:42.513509 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48aecb-02ec-4a09-b4c5-55fe746f280c None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:42.515312 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2c48aecb-02ec-4a09-b4c5-55fe746f280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:42.536473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48aecb-02ec-4a09-b4c5-55fe746f280c None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:42.560469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:42.565564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:42.566230 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:42.566396 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] PATH_INFO: `/v3/projects/3c641cc60c3c4e21add812894e369f35` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:42.569252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c641cc60c3c4e21add812894e369f35)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:42.574928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:42.725342 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c641cc60c3c4e21add812894e369f35', 'request_id': 'req-24955e4a-77bf-47d4-b9b2-5512c3963a98'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:42.729898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:42.735221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c641cc60c3c4e21add812894e369f35', 'request_id': 'req-24955e4a-77bf-47d4-b9b2-5512c3963a98'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:42.798423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c641cc60c3c4e21add812894e369f35', 'request_id': 'req-24955e4a-77bf-47d4-b9b2-5512c3963a98'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:42.819187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] The token cache is being invalidate because project 3c641cc60c3c4e21add812894e369f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:07:42.819663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:42.820465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-24955e4a-77bf-47d4-b9b2-5512c3963a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:42.822368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1284/2617] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:07:42 2022] DELETE /identity/v3/projects/3c641cc60c3c4e21add812894e369f35 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:42.827729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003eb9b5-39c8-437b-a916-82d62895d3d0 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:42.828324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003eb9b5-39c8-437b-a916-82d62895d3d0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:42.828324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003eb9b5-39c8-437b-a916-82d62895d3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:42.886383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-003eb9b5-39c8-437b-a916-82d62895d3d0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:42.886383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-003eb9b5-39c8-437b-a916-82d62895d3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:42.995704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1334/2618] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:43.005218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae13a4-d548-4831-a29a-9e80d7c26ca5 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:43.006209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae13a4-d548-4831-a29a-9e80d7c26ca5 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:43.006682 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae13a4-d548-4831-a29a-9e80d7c26ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:43.047678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ddae13a4-d548-4831-a29a-9e80d7c26ca5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:43.049063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ddae13a4-d548-4831-a29a-9e80d7c26ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:43.072076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1285/2619] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:43.075672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003eb9b5-39c8-437b-a916-82d62895d3d0 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:43.088484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003eb9b5-39c8-437b-a916-82d62895d3d0 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:43.095982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9280906-bf7d-4d34-b5ba-58e64cc6d788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:43.096803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9280906-bf7d-4d34-b5ba-58e64cc6d788 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:43.097187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9280906-bf7d-4d34-b5ba-58e64cc6d788 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:43.097611 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9280906-bf7d-4d34-b5ba-58e64cc6d788 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:43.098526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9280906-bf7d-4d34-b5ba-58e64cc6d788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:07:43.098863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9280906-bf7d-4d34-b5ba-58e64cc6d788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:43.101060 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9280906-bf7d-4d34-b5ba-58e64cc6d788 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:43.120070 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1335/2620] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:07:43.124273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae13a4-d548-4831-a29a-9e80d7c26ca5 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:43.135256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae13a4-d548-4831-a29a-9e80d7c26ca5 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:43.141052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02364f-247d-4754-9c86-8128d6f4146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:43.142082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02364f-247d-4754-9c86-8128d6f4146c admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:43.142463 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02364f-247d-4754-9c86-8128d6f4146c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:43.142807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02364f-247d-4754-9c86-8128d6f4146c admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:43.143610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02364f-247d-4754-9c86-8128d6f4146c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:43.146016 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02364f-247d-4754-9c86-8128d6f4146c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:43.309744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1286/2621] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:07:43 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:43.311066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9280906-bf7d-4d34-b5ba-58e64cc6d788 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:43.328796 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9280906-bf7d-4d34-b5ba-58e64cc6d788 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:43.399510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ba52b-15df-4d58-80b0-5b2d58e9937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:43.400209 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ba52b-15df-4d58-80b0-5b2d58e9937a admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:43.400489 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ba52b-15df-4d58-80b0-5b2d58e9937a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:43.400800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ba52b-15df-4d58-80b0-5b2d58e9937a admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:43.402398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ba52b-15df-4d58-80b0-5b2d58e9937a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:43.405274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ba52b-15df-4d58-80b0-5b2d58e9937a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:43.498872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1336/2622] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:07:43 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:43.503028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02364f-247d-4754-9c86-8128d6f4146c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:43.516256 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02364f-247d-4754-9c86-8128d6f4146c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:43.522307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93584860-97b1-49ef-ab1d-02f71c4a1b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:43.523006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93584860-97b1-49ef-ab1d-02f71c4a1b4c admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:43.523298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93584860-97b1-49ef-ab1d-02f71c4a1b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:43.523579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93584860-97b1-49ef-ab1d-02f71c4a1b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:43.524348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93584860-97b1-49ef-ab1d-02f71c4a1b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:43.526428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93584860-97b1-49ef-ab1d-02f71c4a1b4c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:43.539596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1287/2623] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:07:43.544884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ba52b-15df-4d58-80b0-5b2d58e9937a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:43.558339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ba52b-15df-4d58-80b0-5b2d58e9937a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:43.564196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663df01-a522-48b1-8acd-6a4d79615bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:43.565297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663df01-a522-48b1-8acd-6a4d79615bff admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:43.565297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663df01-a522-48b1-8acd-6a4d79615bff admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:43.565550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663df01-a522-48b1-8acd-6a4d79615bff admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:43.566324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663df01-a522-48b1-8acd-6a4d79615bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:43.577916 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663df01-a522-48b1-8acd-6a4d79615bff admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:43.612248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1337/2624] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:43.616587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93584860-97b1-49ef-ab1d-02f71c4a1b4c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:43.631128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93584860-97b1-49ef-ab1d-02f71c4a1b4c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:43.640710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8b839-383d-475e-8728-2c734c6a5253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:43.641710 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8b839-383d-475e-8728-2c734c6a5253 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:43.642068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8b839-383d-475e-8728-2c734c6a5253 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:43.642714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8b839-383d-475e-8728-2c734c6a5253 admin admin] PATH_INFO: `/v3/projects/8428bcaf1800496ab91221bc2b56938f/users/5869bf55678c4a099609b0cbd1799f81/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:43.669106 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8b839-383d-475e-8728-2c734c6a5253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8428bcaf1800496ab91221bc2b56938f, user_id=5869bf55678c4a099609b0cbd1799f81, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:43.671951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8b839-383d-475e-8728-2c734c6a5253 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:43.782063 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1288/2625] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:43 2022] PUT /identity/v3/projects/8428bcaf1800496ab91221bc2b56938f/users/5869bf55678c4a099609b0cbd1799f81/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:43.795614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfeedb-61a5-4e04-9f52-25ae5dd3ecf8 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:43.796131 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfeedb-61a5-4e04-9f52-25ae5dd3ecf8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:43.796623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfeedb-61a5-4e04-9f52-25ae5dd3ecf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:43.805909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:43.841880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-46bfeedb-61a5-4e04-9f52-25ae5dd3ecf8 None None] MFA Rules not processed for user `5869bf55678c4a099609b0cbd1799f81`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:43.841880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-46bfeedb-61a5-4e04-9f52-25ae5dd3ecf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:43.942579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1338/2626] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:43.943336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8b839-383d-475e-8728-2c734c6a5253 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:44.068592 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8b839-383d-475e-8728-2c734c6a5253 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:44.074945 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251f0d-a534-44db-9513-e07e1376054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:44.075930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251f0d-a534-44db-9513-e07e1376054f admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:44.076231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251f0d-a534-44db-9513-e07e1376054f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:44.076514 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251f0d-a534-44db-9513-e07e1376054f admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:44.080671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9251f0d-a534-44db-9513-e07e1376054f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:44.080671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9251f0d-a534-44db-9513-e07e1376054f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:44.168770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1289/2627] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:07:43 2022] POST /identity/v3/projects => generated 377 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:44.172850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfeedb-61a5-4e04-9f52-25ae5dd3ecf8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:44.186127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfeedb-61a5-4e04-9f52-25ae5dd3ecf8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:44.315186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb8cc1-f272-45e1-a4a0-051634213440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:44.315883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb8cc1-f272-45e1-a4a0-051634213440 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:44.316229 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb8cc1-f272-45e1-a4a0-051634213440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:44.316529 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb8cc1-f272-45e1-a4a0-051634213440 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:44.317186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb8cc1-f272-45e1-a4a0-051634213440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:44.319292 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb8cc1-f272-45e1-a4a0-051634213440 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:44.866936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1339/2628] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:07:44 2022] POST /identity/v3/users => generated 289 bytes in 695 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:07:44.871992 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251f0d-a534-44db-9513-e07e1376054f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:44.884376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251f0d-a534-44db-9513-e07e1376054f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:44.889796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b03a9d-8071-456c-8720-64e371062dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:44.890479 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b03a9d-8071-456c-8720-64e371062dfe admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:44.890770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b03a9d-8071-456c-8720-64e371062dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:44.891039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b03a9d-8071-456c-8720-64e371062dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:44.891778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b03a9d-8071-456c-8720-64e371062dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:44.893441 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b03a9d-8071-456c-8720-64e371062dfe admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:44.909536 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1290/2629] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:44.914065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb8cc1-f272-45e1-a4a0-051634213440 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:44.927077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb8cc1-f272-45e1-a4a0-051634213440 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:44.942771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd13ba6-f863-4ad9-97d9-7f3bc2e05d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:44.942771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd13ba6-f863-4ad9-97d9-7f3bc2e05d3e admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:44.942771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd13ba6-f863-4ad9-97d9-7f3bc2e05d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:44.943394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd13ba6-f863-4ad9-97d9-7f3bc2e05d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:44.944138 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd13ba6-f863-4ad9-97d9-7f3bc2e05d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:44.946194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd13ba6-f863-4ad9-97d9-7f3bc2e05d3e admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:44.959785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1340/2630] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:44.965010 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b03a9d-8071-456c-8720-64e371062dfe admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:44.978421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b03a9d-8071-456c-8720-64e371062dfe admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:44.984679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba140e-35e6-4514-81e2-df694a1f4d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:44.986365 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba140e-35e6-4514-81e2-df694a1f4d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:44.986925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba140e-35e6-4514-81e2-df694a1f4d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:44.987373 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba140e-35e6-4514-81e2-df694a1f4d9e admin admin] PATH_INFO: `/v3/projects/285225bbd61540c3a2150f8f1441df39/users/e2f8d767d149449dbd37cf9335797e71/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:45.008693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba140e-35e6-4514-81e2-df694a1f4d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=285225bbd61540c3a2150f8f1441df39, user_id=e2f8d767d149449dbd37cf9335797e71, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:45.013852 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba140e-35e6-4514-81e2-df694a1f4d9e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:45.104088 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1291/2631] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:44 2022] PUT /identity/v3/projects/285225bbd61540c3a2150f8f1441df39/users/e2f8d767d149449dbd37cf9335797e71/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:07:45.115787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf1d47-ac19-4405-8225-e4eaa3d5ca8c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:45.116072 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf1d47-ac19-4405-8225-e4eaa3d5ca8c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:45.116355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf1d47-ac19-4405-8225-e4eaa3d5ca8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:45.148728 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-3fbf1d47-ac19-4405-8225-e4eaa3d5ca8c None None] MFA Rules not processed for user `e2f8d767d149449dbd37cf9335797e71`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:45.149960 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3fbf1d47-ac19-4405-8225-e4eaa3d5ca8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:45.229015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:45.236808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1341/2632] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:45.240930 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba140e-35e6-4514-81e2-df694a1f4d9e admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:45.305654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:45.375973 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba140e-35e6-4514-81e2-df694a1f4d9e admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:45.396612 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1537-8724-4a1f-a4af-04f83616d092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:45.397996 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1537-8724-4a1f-a4af-04f83616d092 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:45.398534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1537-8724-4a1f-a4af-04f83616d092 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:45.399044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1537-8724-4a1f-a4af-04f83616d092 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:45.400434 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1537-8724-4a1f-a4af-04f83616d092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:45.406097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1537-8724-4a1f-a4af-04f83616d092 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:45.629681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1292/2633] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:07:45 2022] POST /identity/v3/projects => generated 377 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:07:45.634195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf1d47-ac19-4405-8225-e4eaa3d5ca8c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:45.655438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf1d47-ac19-4405-8225-e4eaa3d5ca8c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:45.738154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bfd44-14d7-4074-9343-7a7dbf7842a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:45.739334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bfd44-14d7-4074-9343-7a7dbf7842a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:45.739835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bfd44-14d7-4074-9343-7a7dbf7842a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:45.740331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bfd44-14d7-4074-9343-7a7dbf7842a3 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:45.741793 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bfd44-14d7-4074-9343-7a7dbf7842a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:45.746139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bfd44-14d7-4074-9343-7a7dbf7842a3 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:45.883197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1342/2634] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:07:45 2022] POST /identity/v3/users => generated 295 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:07:45.884913 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1537-8724-4a1f-a4af-04f83616d092 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:45.896015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1537-8724-4a1f-a4af-04f83616d092 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:45.900882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c61d69-b168-485a-a47c-5daf48586979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:45.902094 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c61d69-b168-485a-a47c-5daf48586979 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:45.902382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c61d69-b168-485a-a47c-5daf48586979 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:45.902675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c61d69-b168-485a-a47c-5daf48586979 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:45.903460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c61d69-b168-485a-a47c-5daf48586979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:45.905164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c61d69-b168-485a-a47c-5daf48586979 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:45.918120 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1293/2635] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:07:45.922952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bfd44-14d7-4074-9343-7a7dbf7842a3 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:45.934812 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bfd44-14d7-4074-9343-7a7dbf7842a3 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:45.939696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ac4b7-cedc-46a5-bee9-588e07930b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:45.940609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ac4b7-cedc-46a5-bee9-588e07930b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:45.940921 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ac4b7-cedc-46a5-bee9-588e07930b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:45.941200 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ac4b7-cedc-46a5-bee9-588e07930b73 admin admin] PATH_INFO: `/v3/projects/2cfee8dfdb4a43aca04c12c115a845e4/users/cc53085199d4405f847e40e5d3df18e3/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:45.958448 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ac4b7-cedc-46a5-bee9-588e07930b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfee8dfdb4a43aca04c12c115a845e4, user_id=cc53085199d4405f847e40e5d3df18e3, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:45.961388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ac4b7-cedc-46a5-bee9-588e07930b73 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:46.064747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1343/2636] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:45 2022] PUT /identity/v3/projects/2cfee8dfdb4a43aca04c12c115a845e4/users/cc53085199d4405f847e40e5d3df18e3/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:46.068689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c61d69-b168-485a-a47c-5daf48586979 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:46.078795 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c61d69-b168-485a-a47c-5daf48586979 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:46.147421 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d9a72-838f-46d4-8493-d25bd13ffc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:46.148156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d9a72-838f-46d4-8493-d25bd13ffc86 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:46.148448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d9a72-838f-46d4-8493-d25bd13ffc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:46.148758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d9a72-838f-46d4-8493-d25bd13ffc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:46.153889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d9a72-838f-46d4-8493-d25bd13ffc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:46.153889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d9a72-838f-46d4-8493-d25bd13ffc86 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:46.163700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1294/2637] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:07:46.167985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ac4b7-cedc-46a5-bee9-588e07930b73 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:46.179478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ac4b7-cedc-46a5-bee9-588e07930b73 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:46.185222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4e74-fdfe-493c-8e7c-95e70c08deb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:46.186935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4e74-fdfe-493c-8e7c-95e70c08deb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:46.186935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4e74-fdfe-493c-8e7c-95e70c08deb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:46.186935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4e74-fdfe-493c-8e7c-95e70c08deb0 admin admin] PATH_INFO: `/v3/projects/2cfee8dfdb4a43aca04c12c115a845e4/users/cc53085199d4405f847e40e5d3df18e3/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:46.189084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e4e74-fdfe-493c-8e7c-95e70c08deb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfee8dfdb4a43aca04c12c115a845e4, user_id=cc53085199d4405f847e40e5d3df18e3, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:46.191784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e4e74-fdfe-493c-8e7c-95e70c08deb0 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:46.203477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1344/2638] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:07:46 2022] PUT /identity/v3/projects/2cfee8dfdb4a43aca04c12c115a845e4/users/cc53085199d4405f847e40e5d3df18e3/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:07:46.214950 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad031e-215b-421e-949f-68e0b119cd3f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:46.215243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad031e-215b-421e-949f-68e0b119cd3f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:46.215533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad031e-215b-421e-949f-68e0b119cd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:46.246602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-44ad031e-215b-421e-949f-68e0b119cd3f None None] MFA Rules not processed for user `cc53085199d4405f847e40e5d3df18e3`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:46.248454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-44ad031e-215b-421e-949f-68e0b119cd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:46.327403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1295/2639] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:07:46.339315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4e74-fdfe-493c-8e7c-95e70c08deb0 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:46.343830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-662e4e74-fdfe-493c-8e7c-95e70c08deb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:46.412490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4e74-fdfe-493c-8e7c-95e70c08deb0 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:46.416320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d56ef-0e0d-4cb1-b7a5-469e28e4149a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:46.417680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d56ef-0e0d-4cb1-b7a5-469e28e4149a service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:46.417973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d56ef-0e0d-4cb1-b7a5-469e28e4149a service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:46.418260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d56ef-0e0d-4cb1-b7a5-469e28e4149a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:46.498658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d56ef-0e0d-4cb1-b7a5-469e28e4149a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:46.500253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d56ef-0e0d-4cb1-b7a5-469e28e4149a service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:46.510902 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1345/2640] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:47.957951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad031e-215b-421e-949f-68e0b119cd3f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:47.961671 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-44ad031e-215b-421e-949f-68e0b119cd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:47.991863 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:48.024119 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad031e-215b-421e-949f-68e0b119cd3f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:48.028212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:48.028915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:48.029225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:48.029501 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:48.031507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:48.084069 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d56ef-0e0d-4cb1-b7a5-469e28e4149a service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:48.087494 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c68d56ef-0e0d-4cb1-b7a5-469e28e4149a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:48.100347 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:48.123964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:48.125812 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:48.137783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1296/2641] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:48.188864 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d56ef-0e0d-4cb1-b7a5-469e28e4149a service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:48.195371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176d179-fc86-4c00-9147-2f9df27b0771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:48.196145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176d179-fc86-4c00-9147-2f9df27b0771 service nova] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:48.196506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176d179-fc86-4c00-9147-2f9df27b0771 service nova] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:48.196872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176d179-fc86-4c00-9147-2f9df27b0771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:48.209340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176d179-fc86-4c00-9147-2f9df27b0771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:48.211151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176d179-fc86-4c00-9147-2f9df27b0771 service nova] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:48.223672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1346/2642] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:48.884903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:48.887412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:48.902491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028849f-44d5-4749-87de-fb61e7c73523 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:48.907231 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b3556-5adc-4baf-b9d7-de41edc64d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:48.908245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b3556-5adc-4baf-b9d7-de41edc64d8f service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:48.908798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b3556-5adc-4baf-b9d7-de41edc64d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:48.909401 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b3556-5adc-4baf-b9d7-de41edc64d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:49.014158 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b3556-5adc-4baf-b9d7-de41edc64d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:49.016583 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b3556-5adc-4baf-b9d7-de41edc64d8f service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:49.028378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1297/2643] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:07:51.721751 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176d179-fc86-4c00-9147-2f9df27b0771 service nova] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:51.737882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176d179-fc86-4c00-9147-2f9df27b0771 service nova] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:51.745928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce5107-64d1-480a-9234-839a31c16c18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:51.752205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce5107-64d1-480a-9234-839a31c16c18 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:51.752205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce5107-64d1-480a-9234-839a31c16c18 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:51.752205 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce5107-64d1-480a-9234-839a31c16c18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:51.764159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce5107-64d1-480a-9234-839a31c16c18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:51.767368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce5107-64d1-480a-9234-839a31c16c18 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:51.807200 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1347/2644] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:55.780334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a0c8-32dd-478d-9f5c-650acaef2d40 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:55.781228 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a0c8-32dd-478d-9f5c-650acaef2d40 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:55.781400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a0c8-32dd-478d-9f5c-650acaef2d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:55.813320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-c360a0c8-32dd-478d-9f5c-650acaef2d40 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:55.814968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c360a0c8-32dd-478d-9f5c-650acaef2d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:55.898198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1298/2645] 10.222.0.17 () {58 vars in 967 bytes} [Sun Jul 24 20:07:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:07:55.926720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce5107-64d1-480a-9234-839a31c16c18 service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:55.928302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-46ce5107-64d1-480a-9234-839a31c16c18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:56.027954 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce5107-64d1-480a-9234-839a31c16c18 service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:56.033690 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6ef1e-5b25-4d03-b6c7-2c40d91db8a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:56.034418 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ef1e-5b25-4d03-b6c7-2c40d91db8a0 service swift] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:56.034716 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ef1e-5b25-4d03-b6c7-2c40d91db8a0 service swift] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:56.034997 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ef1e-5b25-4d03-b6c7-2c40d91db8a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:56.044852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6ef1e-5b25-4d03-b6c7-2c40d91db8a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:56.046876 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6ef1e-5b25-4d03-b6c7-2c40d91db8a0 service swift] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:56.057247 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1348/2646] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:07:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:07:56.945987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360a0c8-32dd-478d-9f5c-650acaef2d40 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:56.978339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:57.036045 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360a0c8-32dd-478d-9f5c-650acaef2d40 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:57.040901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:57.041729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:57.042018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:57.042299 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] PATH_INFO: `/v3/users/5869bf55678c4a099609b0cbd1799f81` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:57.043528 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5869bf55678c4a099609b0cbd1799f81)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:57.045523 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:57.335758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5869bf55678c4a099609b0cbd1799f81'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:57.344686 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5869bf55678c4a099609b0cbd1799f81'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:57.345552 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5869bf55678c4a099609b0cbd1799f81'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:57.479471 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5869bf55678c4a099609b0cbd1799f81'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:57.496567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5869bf55678c4a099609b0cbd1799f81'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:57.505722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1299/2647] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:07:56 2022] DELETE /identity/v3/users/5869bf55678c4a099609b0cbd1799f81 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:57.512274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd267b-a97d-434a-b53d-3df8ddaa1925 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:57.513091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd267b-a97d-434a-b53d-3df8ddaa1925 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:57.513091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd267b-a97d-434a-b53d-3df8ddaa1925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:57.537234 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-d8cd267b-a97d-434a-b53d-3df8ddaa1925 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:07:57.538674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d8cd267b-a97d-434a-b53d-3df8ddaa1925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:57.616167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1349/2648] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:07:57.626910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:57.629400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:57.691699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1f987-7c30-4caf-a0a6-415908c78943 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:57.695462 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617191ba-2e76-43d8-912e-26520487231d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:57.696152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617191ba-2e76-43d8-912e-26520487231d service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:57.696442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617191ba-2e76-43d8-912e-26520487231d service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:57.696810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617191ba-2e76-43d8-912e-26520487231d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:57.709831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617191ba-2e76-43d8-912e-26520487231d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:57.709831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617191ba-2e76-43d8-912e-26520487231d service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:57.719843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1300/2649] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:07:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:07:58.473485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd267b-a97d-434a-b53d-3df8ddaa1925 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:58.473485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d8cd267b-a97d-434a-b53d-3df8ddaa1925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:58.485330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd267b-a97d-434a-b53d-3df8ddaa1925 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:58.491360 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:58.493051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:58.493516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:58.494770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] PATH_INFO: `/v3/projects/8428bcaf1800496ab91221bc2b56938f` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:58.498097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8428bcaf1800496ab91221bc2b56938f)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:58.500838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:58.504715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=100 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:58.727109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8428bcaf1800496ab91221bc2b56938f', 'request_id': 'req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:58.738759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8428bcaf1800496ab91221bc2b56938f', 'request_id': 'req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:58.746306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8428bcaf1800496ab91221bc2b56938f', 'request_id': 'req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:58.851817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] The token cache is being invalidate because project 8428bcaf1800496ab91221bc2b56938f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:07:58.852114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:58.852789 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:58.855065 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1350/2650] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:07:58 2022] DELETE /identity/v3/projects/8428bcaf1800496ab91221bc2b56938f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:58.859300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617191ba-2e76-43d8-912e-26520487231d service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:58.869650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617191ba-2e76-43d8-912e-26520487231d service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:58.916367 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:07:58.933277 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:58.934435 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:58.934719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:58.935001 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] PATH_INFO: `/v3/users/e2f8d767d149449dbd37cf9335797e71` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:58.936353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f8d767d149449dbd37cf9335797e71)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:58.938398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:07:59.223512 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f8d767d149449dbd37cf9335797e71'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:59.230383 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f8d767d149449dbd37cf9335797e71'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:59.231132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f8d767d149449dbd37cf9335797e71'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:59.306344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f8d767d149449dbd37cf9335797e71'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:59.320801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f8d767d149449dbd37cf9335797e71'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:07:59.327995 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1301/2651] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:07:58 2022] DELETE /identity/v3/users/e2f8d767d149449dbd37cf9335797e71 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:07:59.797559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:07:59.799177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:07:59.814601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cceaf-a9fc-4502-83d3-bcd251a2d9b7 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:07:59.886765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:07:59.896770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:07:59.896770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:07:59.896770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] PATH_INFO: `/v3/projects/285225bbd61540c3a2150f8f1441df39` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:07:59.896770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=285225bbd61540c3a2150f8f1441df39)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:07:59.896770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:00.029939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285225bbd61540c3a2150f8f1441df39', 'request_id': 'req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.036286 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285225bbd61540c3a2150f8f1441df39', 'request_id': 'req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.046863 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285225bbd61540c3a2150f8f1441df39', 'request_id': 'req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.133686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] The token cache is being invalidate because project 285225bbd61540c3a2150f8f1441df39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:08:00.134217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.135303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.140142 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1351/2652] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:07:59 2022] DELETE /identity/v3/projects/285225bbd61540c3a2150f8f1441df39 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:08:00.143780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:00.145549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:00.159569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28d55c-a9dd-4760-8c5c-9344c06d7497 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:00.216911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:00.217793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:00.218081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:00.218358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] PATH_INFO: `/v3/users/cc53085199d4405f847e40e5d3df18e3` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:00.219579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc53085199d4405f847e40e5d3df18e3)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:00.221628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:00.492965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc53085199d4405f847e40e5d3df18e3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.500364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc53085199d4405f847e40e5d3df18e3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.501156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc53085199d4405f847e40e5d3df18e3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.586971 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc53085199d4405f847e40e5d3df18e3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.589921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:08:00.603198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-14719513-ff8a-47a0-b7f1-79cf2f0f9ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc53085199d4405f847e40e5d3df18e3'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:00.609882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1302/2653] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:08:00 2022] DELETE /identity/v3/users/cc53085199d4405f847e40e5d3df18e3 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:08:01.096723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:01.099650 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:01.116119 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e09b7b-48cd-4edb-8d4e-bb34bfbed87c admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:01.144689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:08:01.210987 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:01.211909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:01.212199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:01.212482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] PATH_INFO: `/v3/projects/2cfee8dfdb4a43aca04c12c115a845e4` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:01.213985 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cfee8dfdb4a43aca04c12c115a845e4)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:01.216282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:01.368482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cfee8dfdb4a43aca04c12c115a845e4', 'request_id': 'req-04fc2822-d1d6-49b4-b95e-03b94238b51c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:01.376548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cfee8dfdb4a43aca04c12c115a845e4', 'request_id': 'req-04fc2822-d1d6-49b4-b95e-03b94238b51c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:01.386359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cfee8dfdb4a43aca04c12c115a845e4', 'request_id': 'req-04fc2822-d1d6-49b4-b95e-03b94238b51c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:01.468238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] The token cache is being invalidate because project 2cfee8dfdb4a43aca04c12c115a845e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:08:01.468704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:01.469872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-04fc2822-d1d6-49b4-b95e-03b94238b51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:01.475845 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1352/2654] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:08:01 2022] DELETE /identity/v3/projects/2cfee8dfdb4a43aca04c12c115a845e4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:08:01.477557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5327ab-302c-43c2-8504-dbdde3763a28 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:01.478031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5327ab-302c-43c2-8504-dbdde3763a28 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:01.481517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5327ab-302c-43c2-8504-dbdde3763a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:01.497869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-fd5327ab-302c-43c2-8504-dbdde3763a28 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:08:01.499114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fd5327ab-302c-43c2-8504-dbdde3763a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:01.577456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1303/2655] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:08:01.585541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f28583-b953-432b-92ca-805d12302af3 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:01.585924 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f28583-b953-432b-92ca-805d12302af3 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:01.586271 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f28583-b953-432b-92ca-805d12302af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:01.608913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c9f28583-b953-432b-92ca-805d12302af3 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:08:01.609938 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c9f28583-b953-432b-92ca-805d12302af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:01.618377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1353/2656] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:08:01.623864 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5327ab-302c-43c2-8504-dbdde3763a28 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:01.634351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5327ab-302c-43c2-8504-dbdde3763a28 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:01.639669 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1759c9-c390-42a4-a1df-f6b649b14aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:01.640334 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1759c9-c390-42a4-a1df-f6b649b14aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:01.640619 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1759c9-c390-42a4-a1df-f6b649b14aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:01.640922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1759c9-c390-42a4-a1df-f6b649b14aa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:01.641652 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1759c9-c390-42a4-a1df-f6b649b14aa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:08:01.641960 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1759c9-c390-42a4-a1df-f6b649b14aa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:01.643580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1759c9-c390-42a4-a1df-f6b649b14aa0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:01.655807 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1304/2657] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:08:01.660279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f28583-b953-432b-92ca-805d12302af3 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:01.670022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f28583-b953-432b-92ca-805d12302af3 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:01.674653 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afba87e-5062-4320-89d0-75646921078a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:01.675307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afba87e-5062-4320-89d0-75646921078a admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:01.675599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afba87e-5062-4320-89d0-75646921078a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:01.675906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afba87e-5062-4320-89d0-75646921078a admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:01.676599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afba87e-5062-4320-89d0-75646921078a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:01.678591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afba87e-5062-4320-89d0-75646921078a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:01.920190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1354/2658] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:08:01 2022] POST /identity/v3/projects => generated 385 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:08:01.921819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1759c9-c390-42a4-a1df-f6b649b14aa0 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:01.940884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1759c9-c390-42a4-a1df-f6b649b14aa0 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:02.025921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13725fe9-f0e5-4fcb-84de-f1bbf7dad336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:02.026672 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13725fe9-f0e5-4fcb-84de-f1bbf7dad336 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:02.026966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13725fe9-f0e5-4fcb-84de-f1bbf7dad336 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:02.027250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13725fe9-f0e5-4fcb-84de-f1bbf7dad336 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:02.027956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13725fe9-f0e5-4fcb-84de-f1bbf7dad336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:02.029975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13725fe9-f0e5-4fcb-84de-f1bbf7dad336 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:02.335965 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1305/2659] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:08:01 2022] POST /identity/v3/users => generated 293 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:08:02.339804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afba87e-5062-4320-89d0-75646921078a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:02.351392 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afba87e-5062-4320-89d0-75646921078a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:02.356837 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710bb7e-00b8-4b87-829e-0713c575fd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:02.357534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710bb7e-00b8-4b87-829e-0713c575fd56 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:02.357823 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710bb7e-00b8-4b87-829e-0713c575fd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:02.358107 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710bb7e-00b8-4b87-829e-0713c575fd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:02.358879 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710bb7e-00b8-4b87-829e-0713c575fd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:02.360502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710bb7e-00b8-4b87-829e-0713c575fd56 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:02.504269 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1355/2660] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:08:02.508911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13725fe9-f0e5-4fcb-84de-f1bbf7dad336 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:02.520768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13725fe9-f0e5-4fcb-84de-f1bbf7dad336 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:02.525701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b677b-1629-4508-a5cf-49856241fd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:02.526910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b677b-1629-4508-a5cf-49856241fd79 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:02.527198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b677b-1629-4508-a5cf-49856241fd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:02.527478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b677b-1629-4508-a5cf-49856241fd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:02.528227 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b677b-1629-4508-a5cf-49856241fd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:02.529921 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b677b-1629-4508-a5cf-49856241fd79 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:02.542822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1306/2661] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:08:02.549242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710bb7e-00b8-4b87-829e-0713c575fd56 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:02.560541 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710bb7e-00b8-4b87-829e-0713c575fd56 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:02.566352 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8ee56-4c0c-4f3b-bae1-f09bd73a1b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:02.567465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8ee56-4c0c-4f3b-bae1-f09bd73a1b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:02.567848 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8ee56-4c0c-4f3b-bae1-f09bd73a1b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:02.568475 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8ee56-4c0c-4f3b-bae1-f09bd73a1b2d admin admin] PATH_INFO: `/v3/projects/27151bcd0df34f5593f7a290b76794a9/users/89c6b37e9c924d77988c45c5e110b297/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:02.591199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8ee56-4c0c-4f3b-bae1-f09bd73a1b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27151bcd0df34f5593f7a290b76794a9, user_id=89c6b37e9c924d77988c45c5e110b297, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:02.593989 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8ee56-4c0c-4f3b-bae1-f09bd73a1b2d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:02.679350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1356/2662] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:08:02 2022] PUT /identity/v3/projects/27151bcd0df34f5593f7a290b76794a9/users/89c6b37e9c924d77988c45c5e110b297/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:08:02.691520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d50de-1c7f-48b1-af43-052a3c7a7492 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:02.691851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d50de-1c7f-48b1-af43-052a3c7a7492 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:02.692137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d50de-1c7f-48b1-af43-052a3c7a7492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:02.725968 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-298d50de-1c7f-48b1-af43-052a3c7a7492 None None] MFA Rules not processed for user `89c6b37e9c924d77988c45c5e110b297`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:08:02.727198 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-298d50de-1c7f-48b1-af43-052a3c7a7492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:02.766796 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:08:02.808517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1307/2663] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:08:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:08:02.824568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8ee56-4c0c-4f3b-bae1-f09bd73a1b2d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:02.826643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-77a8ee56-4c0c-4f3b-bae1-f09bd73a1b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:02.890209 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:08:02.894912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8ee56-4c0c-4f3b-bae1-f09bd73a1b2d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:02.899039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc9de3-5ff5-4579-9161-cd997e166f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:02.899768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc9de3-5ff5-4579-9161-cd997e166f9c service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:02.900095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc9de3-5ff5-4579-9161-cd997e166f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:02.906915 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc9de3-5ff5-4579-9161-cd997e166f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:02.910075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc9de3-5ff5-4579-9161-cd997e166f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:02.911525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc9de3-5ff5-4579-9161-cd997e166f9c service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:02.922486 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1357/2664] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:08:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:08:06.251689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d50de-1c7f-48b1-af43-052a3c7a7492 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:06.326442 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d50de-1c7f-48b1-af43-052a3c7a7492 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:06.331087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:06.331981 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:06.332266 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:06.332546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] PATH_INFO: `/v3/users/89c6b37e9c924d77988c45c5e110b297` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:06.333816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c6b37e9c924d77988c45c5e110b297)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:06.335743 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:06.646650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c6b37e9c924d77988c45c5e110b297'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:06.654330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c6b37e9c924d77988c45c5e110b297'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:06.655098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c6b37e9c924d77988c45c5e110b297'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:06.752288 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c6b37e9c924d77988c45c5e110b297'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:06.765972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c6b37e9c924d77988c45c5e110b297'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:06.772797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1308/2665] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:08:06 2022] DELETE /identity/v3/users/89c6b37e9c924d77988c45c5e110b297 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:08:06.777605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5fb21-fd64-4755-85b5-193cd7d445cb None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:06.778095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5fb21-fd64-4755-85b5-193cd7d445cb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:06.778194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5fb21-fd64-4755-85b5-193cd7d445cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:06.796963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-05c5fb21-fd64-4755-85b5-193cd7d445cb None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:08:06.798193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-05c5fb21-fd64-4755-85b5-193cd7d445cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:06.872480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1358/2666] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:08:06.882918 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:06.884344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:06.958572 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97045f8c-9bb7-4555-9a41-f55a1fafd585 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:06.965423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b11f2-68b9-4e86-bd7e-e9ffd0be9c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:06.966579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b11f2-68b9-4e86-bd7e-e9ffd0be9c45 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:06.967116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b11f2-68b9-4e86-bd7e-e9ffd0be9c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:06.967615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b11f2-68b9-4e86-bd7e-e9ffd0be9c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:06.984196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b11f2-68b9-4e86-bd7e-e9ffd0be9c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:06.987125 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b11f2-68b9-4e86-bd7e-e9ffd0be9c45 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:06.995091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:08:07.007000 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1309/2667] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:08:07.489283 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5fb21-fd64-4755-85b5-193cd7d445cb None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:07.490799 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-05c5fb21-fd64-4755-85b5-193cd7d445cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:07.504627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5fb21-fd64-4755-85b5-193cd7d445cb None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:07.511353 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:07.512296 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:07.512588 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:07.512889 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] PATH_INFO: `/v3/projects/27151bcd0df34f5593f7a290b76794a9` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:07.514136 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27151bcd0df34f5593f7a290b76794a9)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:07.516527 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:07.659998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27151bcd0df34f5593f7a290b76794a9', 'request_id': 'req-9507d586-9ecb-4ba6-816e-3aa91f8b7143'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:07.665465 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27151bcd0df34f5593f7a290b76794a9', 'request_id': 'req-9507d586-9ecb-4ba6-816e-3aa91f8b7143'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:07.672227 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27151bcd0df34f5593f7a290b76794a9', 'request_id': 'req-9507d586-9ecb-4ba6-816e-3aa91f8b7143'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:07.771748 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] The token cache is being invalidate because project 27151bcd0df34f5593f7a290b76794a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:08:07.771983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:07.772578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:07.774158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1359/2668] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:08:07 2022] DELETE /identity/v3/projects/27151bcd0df34f5593f7a290b76794a9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:08:17.297976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:08:17.768478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:08:41.634558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b11f2-68b9-4e86-bd7e-e9ffd0be9c45 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:41.643956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b11f2-68b9-4e86-bd7e-e9ffd0be9c45 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:41.694576 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029499d-f092-4036-b42b-3cee8dcfdb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:41.695156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029499d-f092-4036-b42b-3cee8dcfdb31 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:41.695451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029499d-f092-4036-b42b-3cee8dcfdb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:41.695840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029499d-f092-4036-b42b-3cee8dcfdb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:41.698371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a029499d-f092-4036-b42b-3cee8dcfdb31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:41.769388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029499d-f092-4036-b42b-3cee8dcfdb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:41.770840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029499d-f092-4036-b42b-3cee8dcfdb31 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:41.781114 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1310/2669] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:08:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:08:45.541569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:45.543111 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:45.556912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d586-9ecb-4ba6-816e-3aa91f8b7143 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:45.562680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:45.571871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:45.571871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:45.571871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] PATH_INFO: `/v3/users/55ed193c93c446a8b373f770512e5160` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:45.571871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ed193c93c446a8b373f770512e5160)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:45.571871 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:45.735422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ed193c93c446a8b373f770512e5160'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:45.748255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ed193c93c446a8b373f770512e5160'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:45.755148 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ed193c93c446a8b373f770512e5160'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:45.755866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ed193c93c446a8b373f770512e5160'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:45.854439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-343a4a45-3cf9-47be-96d0-446e37842346 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ed193c93c446a8b373f770512e5160'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:45.861671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1360/2670] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:08:45 2022] DELETE /identity/v3/users/55ed193c93c446a8b373f770512e5160 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:08:46.119856 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029499d-f092-4036-b42b-3cee8dcfdb31 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:46.138407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029499d-f092-4036-b42b-3cee8dcfdb31 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:46.195285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:46.196219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:46.196518 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:46.196818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] PATH_INFO: `/v3/projects/49f6a7894899436d8ce94d9a7500533a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:46.198161 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f6a7894899436d8ce94d9a7500533a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:46.200675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:46.364826 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f6a7894899436d8ce94d9a7500533a', 'request_id': 'req-9a495c9d-b711-43cd-8894-75aa99b81349'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:46.366611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:08:46.371415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f6a7894899436d8ce94d9a7500533a', 'request_id': 'req-9a495c9d-b711-43cd-8894-75aa99b81349'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:46.495495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f6a7894899436d8ce94d9a7500533a', 'request_id': 'req-9a495c9d-b711-43cd-8894-75aa99b81349'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:46.508309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] The token cache is being invalidate because project 49f6a7894899436d8ce94d9a7500533a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:08:46.508627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:46.509368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-9a495c9d-b711-43cd-8894-75aa99b81349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:08:46.511448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1311/2671] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:08:46 2022] DELETE /identity/v3/projects/49f6a7894899436d8ce94d9a7500533a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:08:46.517586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794652-a746-4a42-ac8c-2b5476cc6c4b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:46.518030 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794652-a746-4a42-ac8c-2b5476cc6c4b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:46.518322 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794652-a746-4a42-ac8c-2b5476cc6c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:46.537901 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f0794652-a746-4a42-ac8c-2b5476cc6c4b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:08:46.539178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f0794652-a746-4a42-ac8c-2b5476cc6c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:46.603943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1361/2672] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:08:46.612565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbeda5-8168-48a7-a22f-599403951dd0 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:46.612900 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbeda5-8168-48a7-a22f-599403951dd0 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:46.613294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbeda5-8168-48a7-a22f-599403951dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:46.634135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-30cbeda5-8168-48a7-a22f-599403951dd0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:08:46.635289 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-30cbeda5-8168-48a7-a22f-599403951dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:46.646006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1312/2673] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:08:46.650500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0794652-a746-4a42-ac8c-2b5476cc6c4b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:46.663667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0794652-a746-4a42-ac8c-2b5476cc6c4b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:46.670681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacabb01-09c2-494e-90b4-f35c084f6c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:46.671674 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacabb01-09c2-494e-90b4-f35c084f6c71 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:46.672099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacabb01-09c2-494e-90b4-f35c084f6c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:46.672511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacabb01-09c2-494e-90b4-f35c084f6c71 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:46.673737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacabb01-09c2-494e-90b4-f35c084f6c71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:08:46.674208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacabb01-09c2-494e-90b4-f35c084f6c71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:46.676705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacabb01-09c2-494e-90b4-f35c084f6c71 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:46.696340 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1362/2674] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 24 20:08:46.702051 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbeda5-8168-48a7-a22f-599403951dd0 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:46.712129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbeda5-8168-48a7-a22f-599403951dd0 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:46.721464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d156-f1c1-42f0-ba26-ba677b61217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:46.722164 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d156-f1c1-42f0-ba26-ba677b61217a admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:46.722452 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d156-f1c1-42f0-ba26-ba677b61217a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:46.722735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d156-f1c1-42f0-ba26-ba677b61217a admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:46.723448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d156-f1c1-42f0-ba26-ba677b61217a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:46.725912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d156-f1c1-42f0-ba26-ba677b61217a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:46.850009 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1313/2675] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:08:46 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:08:46.851881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacabb01-09c2-494e-90b4-f35c084f6c71 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:46.862177 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacabb01-09c2-494e-90b4-f35c084f6c71 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:46.924075 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d74d1-1b03-4cce-8d17-cf8978f6fdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:46.924806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d74d1-1b03-4cce-8d17-cf8978f6fdd4 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:46.925096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d74d1-1b03-4cce-8d17-cf8978f6fdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:46.925372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d74d1-1b03-4cce-8d17-cf8978f6fdd4 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:46.926086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d74d1-1b03-4cce-8d17-cf8978f6fdd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:46.928130 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d74d1-1b03-4cce-8d17-cf8978f6fdd4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:46.936165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:08:47.010675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1363/2676] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:08:46 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:08:47.015095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d156-f1c1-42f0-ba26-ba677b61217a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:47.026301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d156-f1c1-42f0-ba26-ba677b61217a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:47.031296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d23ee6-53fc-4b46-8df8-24472d7c4ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:47.031975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d23ee6-53fc-4b46-8df8-24472d7c4ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:47.032262 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d23ee6-53fc-4b46-8df8-24472d7c4ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:47.032541 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d23ee6-53fc-4b46-8df8-24472d7c4ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:47.033413 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d23ee6-53fc-4b46-8df8-24472d7c4ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:47.035037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d23ee6-53fc-4b46-8df8-24472d7c4ea4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:47.046767 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1314/2677] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:08:47.051203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d74d1-1b03-4cce-8d17-cf8978f6fdd4 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:47.062856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d74d1-1b03-4cce-8d17-cf8978f6fdd4 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:47.069356 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e2ad7-9595-4ca0-92ed-cc8602e07feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:47.070035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e2ad7-9595-4ca0-92ed-cc8602e07feb admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:47.070390 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e2ad7-9595-4ca0-92ed-cc8602e07feb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:47.070580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e2ad7-9595-4ca0-92ed-cc8602e07feb admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:47.071312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e2ad7-9595-4ca0-92ed-cc8602e07feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:47.072967 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e2ad7-9595-4ca0-92ed-cc8602e07feb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:47.089193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1364/2678] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:08:47.090068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d23ee6-53fc-4b46-8df8-24472d7c4ea4 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:47.113135 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d23ee6-53fc-4b46-8df8-24472d7c4ea4 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:47.118058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebbaa8-bb7c-4d5d-b963-5ae21874b341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:47.118962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebbaa8-bb7c-4d5d-b963-5ae21874b341 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:47.119248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebbaa8-bb7c-4d5d-b963-5ae21874b341 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:47.119524 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebbaa8-bb7c-4d5d-b963-5ae21874b341 admin admin] PATH_INFO: `/v3/projects/8fac69862bde46c8b40d8a4cc26ca379/users/59e631f2d387466aa2ce6ab2be0414a0/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:47.135988 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebbaa8-bb7c-4d5d-b963-5ae21874b341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fac69862bde46c8b40d8a4cc26ca379, user_id=59e631f2d387466aa2ce6ab2be0414a0, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:47.138769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebbaa8-bb7c-4d5d-b963-5ae21874b341 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:47.236417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1315/2679] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:08:47 2022] PUT /identity/v3/projects/8fac69862bde46c8b40d8a4cc26ca379/users/59e631f2d387466aa2ce6ab2be0414a0/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:08:47.241704 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aba536-5503-4318-96e9-eba1c7afdd6e None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:47.242078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aba536-5503-4318-96e9-eba1c7afdd6e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:47.242312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aba536-5503-4318-96e9-eba1c7afdd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:47.261411 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-26aba536-5503-4318-96e9-eba1c7afdd6e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:08:47.262654 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-26aba536-5503-4318-96e9-eba1c7afdd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:47.331523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1365/2680] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:08:47.342753 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebbaa8-bb7c-4d5d-b963-5ae21874b341 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:47.344791 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ddebbaa8-bb7c-4d5d-b963-5ae21874b341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:47.411868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebbaa8-bb7c-4d5d-b963-5ae21874b341 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:47.415628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b41ce-5df2-408d-95c9-9cba5d0b28a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:47.416313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b41ce-5df2-408d-95c9-9cba5d0b28a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:47.416600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b41ce-5df2-408d-95c9-9cba5d0b28a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:47.416926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b41ce-5df2-408d-95c9-9cba5d0b28a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:47.427955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b41ce-5df2-408d-95c9-9cba5d0b28a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:47.433520 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b41ce-5df2-408d-95c9-9cba5d0b28a6 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:47.440694 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1316/2681] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:08:54.161035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af771cf9-9885-46fc-a11d-41709f9ec109 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:54.161772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af771cf9-9885-46fc-a11d-41709f9ec109 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:54.162246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af771cf9-9885-46fc-a11d-41709f9ec109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:54.207566 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-af771cf9-9885-46fc-a11d-41709f9ec109 None None] MFA Rules not processed for user `59e631f2d387466aa2ce6ab2be0414a0`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:08:54.209091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-af771cf9-9885-46fc-a11d-41709f9ec109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:54.328370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1366/2682] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:08:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:08:54.337642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b41ce-5df2-408d-95c9-9cba5d0b28a6 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:08:54.339344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-5c6b41ce-5df2-408d-95c9-9cba5d0b28a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:54.429002 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b41ce-5df2-408d-95c9-9cba5d0b28a6 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:08:54.437251 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237b9df-dbc2-46a6-a638-5fc02a9f32ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:08:54.438549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237b9df-dbc2-46a6-a638-5fc02a9f32ff service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:54.438835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237b9df-dbc2-46a6-a638-5fc02a9f32ff service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:54.439115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237b9df-dbc2-46a6-a638-5fc02a9f32ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:54.448473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237b9df-dbc2-46a6-a638-5fc02a9f32ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:08:54.450690 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237b9df-dbc2-46a6-a638-5fc02a9f32ff service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:08:54.461823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1317/2683] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:08:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:08:57.204991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebc2fe-cf2a-46d3-b4ff-98a0db1f73f5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:08:57.205614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebc2fe-cf2a-46d3-b4ff-98a0db1f73f5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:08:57.206250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebc2fe-cf2a-46d3-b4ff-98a0db1f73f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:08:57.228315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-daebc2fe-cf2a-46d3-b4ff-98a0db1f73f5 None None] MFA Rules not processed for user `59e631f2d387466aa2ce6ab2be0414a0`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:08:57.230018 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-daebc2fe-cf2a-46d3-b4ff-98a0db1f73f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:08:57.242063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1367/2684] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:08:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:01.657005 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237b9df-dbc2-46a6-a638-5fc02a9f32ff service nova] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:01.662546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:01.666547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237b9df-dbc2-46a6-a638-5fc02a9f32ff service nova] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:01.670360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d08c2-dafa-4067-8a64-da310dcc2de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:01.671004 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d08c2-dafa-4067-8a64-da310dcc2de0 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:01.671290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d08c2-dafa-4067-8a64-da310dcc2de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:01.671568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d08c2-dafa-4067-8a64-da310dcc2de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:01.673344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-318d08c2-dafa-4067-8a64-da310dcc2de0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:01.686250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d08c2-dafa-4067-8a64-da310dcc2de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:01.687653 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d08c2-dafa-4067-8a64-da310dcc2de0 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:01.697663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1318/2685] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:07.229880 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:07.607712 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebc2fe-cf2a-46d3-b4ff-98a0db1f73f5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:07.610688 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-daebc2fe-cf2a-46d3-b4ff-98a0db1f73f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:07.629577 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebc2fe-cf2a-46d3-b4ff-98a0db1f73f5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:07.634553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:07.635969 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:07.636255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:07.636554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] PATH_INFO: `/v3/users/822e56f086f0481ca4c6970adf85e396` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:07.638425 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=822e56f086f0481ca4c6970adf85e396)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:07.640347 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:07.846046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '822e56f086f0481ca4c6970adf85e396'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:07.858414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '822e56f086f0481ca4c6970adf85e396'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:07.865424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '822e56f086f0481ca4c6970adf85e396'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:07.866179 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '822e56f086f0481ca4c6970adf85e396'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:07.991867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ff8718f6-9632-4a0d-8260-034cc5bd93d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '822e56f086f0481ca4c6970adf85e396'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:08.001379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1368/2686] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:09:07 2022] DELETE /identity/v3/users/822e56f086f0481ca4c6970adf85e396 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:09:08.421579 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d08c2-dafa-4067-8a64-da310dcc2de0 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:08.437776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d08c2-dafa-4067-8a64-da310dcc2de0 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:08.503804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:08.504737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:08.505033 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:08.505317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] PATH_INFO: `/v3/projects/dcc8f2e8d4e64866a624dcd6d1de5d81` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:08.506703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc8f2e8d4e64866a624dcd6d1de5d81)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:08.508459 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:08.642879 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc8f2e8d4e64866a624dcd6d1de5d81', 'request_id': 'req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:08.650925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc8f2e8d4e64866a624dcd6d1de5d81', 'request_id': 'req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:08.758886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc8f2e8d4e64866a624dcd6d1de5d81', 'request_id': 'req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:08.773855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] The token cache is being invalidate because project dcc8f2e8d4e64866a624dcd6d1de5d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:09:08.774225 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:08.774935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d6b10c3b-4961-469f-b009-b5cc6cdb11dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:08.776564 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1319/2687] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:09:08 2022] DELETE /identity/v3/projects/dcc8f2e8d4e64866a624dcd6d1de5d81 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:08.782543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10aeb9-b364-40f0-bb86-304fb163ba20 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:08.783082 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10aeb9-b364-40f0-bb86-304fb163ba20 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:08.783259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10aeb9-b364-40f0-bb86-304fb163ba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:08.805059 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-af10aeb9-b364-40f0-bb86-304fb163ba20 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:08.806311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-af10aeb9-b364-40f0-bb86-304fb163ba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:08.881298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1369/2688] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:08.889810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf3f67-c893-4cee-ad2c-d0635587e12b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:08.890406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf3f67-c893-4cee-ad2c-d0635587e12b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:08.891189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf3f67-c893-4cee-ad2c-d0635587e12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:08.914721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-afaf3f67-c893-4cee-ad2c-d0635587e12b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:08.916244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-afaf3f67-c893-4cee-ad2c-d0635587e12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:08.931532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1320/2689] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:08.936792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10aeb9-b364-40f0-bb86-304fb163ba20 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:08.947224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10aeb9-b364-40f0-bb86-304fb163ba20 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:08.953182 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c70b25-6895-407b-a563-5b50c09f6eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:08.953935 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c70b25-6895-407b-a563-5b50c09f6eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:08.956775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c70b25-6895-407b-a563-5b50c09f6eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:08.956775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c70b25-6895-407b-a563-5b50c09f6eb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:08.956775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c70b25-6895-407b-a563-5b50c09f6eb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:09:08.956775 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c70b25-6895-407b-a563-5b50c09f6eb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:08.957171 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c70b25-6895-407b-a563-5b50c09f6eb4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:08.973033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1370/2690] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:09:08.976371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf3f67-c893-4cee-ad2c-d0635587e12b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:08.986585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf3f67-c893-4cee-ad2c-d0635587e12b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:08.992813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe0d5b-f9ad-4546-a60f-e902146ea551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:08.993650 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe0d5b-f9ad-4546-a60f-e902146ea551 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:08.993994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe0d5b-f9ad-4546-a60f-e902146ea551 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:08.994339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe0d5b-f9ad-4546-a60f-e902146ea551 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:08.995115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe0d5b-f9ad-4546-a60f-e902146ea551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:08.997540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe0d5b-f9ad-4546-a60f-e902146ea551 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:09.162763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1321/2691] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:09:08 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:09:09.165165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c70b25-6895-407b-a563-5b50c09f6eb4 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:09.175460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c70b25-6895-407b-a563-5b50c09f6eb4 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:09.253526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3764c3-1d60-401f-997e-d9901bf73bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:09.254165 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3764c3-1d60-401f-997e-d9901bf73bbb admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:09.254460 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3764c3-1d60-401f-997e-d9901bf73bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:09.257882 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3764c3-1d60-401f-997e-d9901bf73bbb admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:09.260219 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3764c3-1d60-401f-997e-d9901bf73bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:09.266643 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3764c3-1d60-401f-997e-d9901bf73bbb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:09.282687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:09.400774 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1371/2692] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:09:09 2022] POST /identity/v3/users => generated 287 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:09:09.406015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe0d5b-f9ad-4546-a60f-e902146ea551 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:09.438663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe0d5b-f9ad-4546-a60f-e902146ea551 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:09.462392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435c709-2b87-48c4-89c8-c8b019cbeb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:09.464138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435c709-2b87-48c4-89c8-c8b019cbeb7c admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:09.478240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435c709-2b87-48c4-89c8-c8b019cbeb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:09.478240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435c709-2b87-48c4-89c8-c8b019cbeb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:09.478240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435c709-2b87-48c4-89c8-c8b019cbeb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:09.478240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435c709-2b87-48c4-89c8-c8b019cbeb7c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:09.489077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1322/2693] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:09:09.494972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3764c3-1d60-401f-997e-d9901bf73bbb admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:09.514680 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3764c3-1d60-401f-997e-d9901bf73bbb admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:09.520307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66747e42-c3d5-4cde-96e5-ec5c2051832e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:09.521043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66747e42-c3d5-4cde-96e5-ec5c2051832e admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:09.521332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66747e42-c3d5-4cde-96e5-ec5c2051832e admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:09.521614 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66747e42-c3d5-4cde-96e5-ec5c2051832e admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:09.522434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66747e42-c3d5-4cde-96e5-ec5c2051832e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:09.524118 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66747e42-c3d5-4cde-96e5-ec5c2051832e admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:09.535966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1372/2694] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:09:09.541991 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435c709-2b87-48c4-89c8-c8b019cbeb7c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:09.552869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435c709-2b87-48c4-89c8-c8b019cbeb7c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:09.557497 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546ec69-8908-4463-9ff4-f2f4419d0dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:09.558406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546ec69-8908-4463-9ff4-f2f4419d0dee admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:09.558695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546ec69-8908-4463-9ff4-f2f4419d0dee admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:09.558972 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546ec69-8908-4463-9ff4-f2f4419d0dee admin admin] PATH_INFO: `/v3/projects/31d4d37fc0ee42af995b8717d76810e8/users/ad7191a8ee2d4630b9bddff0403d7688/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:09.576983 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546ec69-8908-4463-9ff4-f2f4419d0dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d4d37fc0ee42af995b8717d76810e8, user_id=ad7191a8ee2d4630b9bddff0403d7688, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:09.579886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546ec69-8908-4463-9ff4-f2f4419d0dee admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:09.729261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1323/2695] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:09:09 2022] PUT /identity/v3/projects/31d4d37fc0ee42af995b8717d76810e8/users/ad7191a8ee2d4630b9bddff0403d7688/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:09:09.745349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a9d74-42bc-4f97-93af-88d050ecff27 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:09.745887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a9d74-42bc-4f97-93af-88d050ecff27 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:09.748011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a9d74-42bc-4f97-93af-88d050ecff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:09.781158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1d3a9d74-42bc-4f97-93af-88d050ecff27 None None] MFA Rules not processed for user `ad7191a8ee2d4630b9bddff0403d7688`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:09.781158 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1d3a9d74-42bc-4f97-93af-88d050ecff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:09.847807 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1373/2696] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:09.860638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546ec69-8908-4463-9ff4-f2f4419d0dee admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:09.863054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1546ec69-8908-4463-9ff4-f2f4419d0dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:09.900261 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:09.991229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546ec69-8908-4463-9ff4-f2f4419d0dee admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:09.999593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c07f6f-b719-4e45-9997-9875db4f227d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:10.000273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c07f6f-b719-4e45-9997-9875db4f227d service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:10.000548 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c07f6f-b719-4e45-9997-9875db4f227d service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:10.000868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c07f6f-b719-4e45-9997-9875db4f227d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:10.009860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c07f6f-b719-4e45-9997-9875db4f227d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:10.012963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c07f6f-b719-4e45-9997-9875db4f227d service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:10.023102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1324/2697] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:10.108195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6fa13-1a7e-4766-8d5d-e860062005d5 None None] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:10.109262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6fa13-1a7e-4766-8d5d-e860062005d5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:10.109792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6fa13-1a7e-4766-8d5d-e860062005d5 None None] PATH_INFO: `/` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:10.112270 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1374/2698] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:09:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:09:10.118975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cee17c-b201-495c-8d3c-59c0cafc5a60 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:10.120024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cee17c-b201-495c-8d3c-59c0cafc5a60 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:10.120260 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cee17c-b201-495c-8d3c-59c0cafc5a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:10.141943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-e6cee17c-b201-495c-8d3c-59c0cafc5a60 None None] MFA Rules not processed for user `b9763f9899214df3929f3816f009e857`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:10.143656 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-e6cee17c-b201-495c-8d3c-59c0cafc5a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:10.219302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1325/2699] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:09:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:10.233739 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6fa13-1a7e-4766-8d5d-e860062005d5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:10.236304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-d0d6fa13-1a7e-4766-8d5d-e860062005d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:10.259595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6fa13-1a7e-4766-8d5d-e860062005d5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:10.263838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86227602-7fed-4d35-9bc6-c252e48968a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:10.264668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86227602-7fed-4d35-9bc6-c252e48968a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:10.265388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86227602-7fed-4d35-9bc6-c252e48968a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:10.265865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86227602-7fed-4d35-9bc6-c252e48968a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:10.274408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86227602-7fed-4d35-9bc6-c252e48968a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:10.275931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86227602-7fed-4d35-9bc6-c252e48968a0 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:10.285215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1375/2700] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:10.375146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7170f8e-7c8c-45f8-aac5-33add233a5c2 None None] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:10.375567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7170f8e-7c8c-45f8-aac5-33add233a5c2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:10.376286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7170f8e-7c8c-45f8-aac5-33add233a5c2 None None] PATH_INFO: `/` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:10.377964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1326/2701] 10.222.0.17 () {54 vars in 921 bytes} [Sun Jul 24 20:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 24 20:09:10.383773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262e894-fb5a-4fb5-9d45-467310d816c5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:10.384095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262e894-fb5a-4fb5-9d45-467310d816c5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:10.384378 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262e894-fb5a-4fb5-9d45-467310d816c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:10.405637 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-3262e894-fb5a-4fb5-9d45-467310d816c5 None None] MFA Rules not processed for user `123b7f659c6a4ceba18ce268b1335542`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:10.407395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3262e894-fb5a-4fb5-9d45-467310d816c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:10.420570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1376/2702] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Jul 24 20:09:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:10.435427 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7170f8e-7c8c-45f8-aac5-33add233a5c2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:10.447923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7170f8e-7c8c-45f8-aac5-33add233a5c2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:10.452205 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b3abc-f68c-4859-97bc-e2180c4942cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:10.452894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b3abc-f68c-4859-97bc-e2180c4942cf service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:10.453180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b3abc-f68c-4859-97bc-e2180c4942cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:10.453461 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b3abc-f68c-4859-97bc-e2180c4942cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:10.464527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b3abc-f68c-4859-97bc-e2180c4942cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:10.466599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b3abc-f68c-4859-97bc-e2180c4942cf service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:10.478407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1327/2703] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:12.913099 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262e894-fb5a-4fb5-9d45-467310d816c5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:12.914554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3262e894-fb5a-4fb5-9d45-467310d816c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:13.009852 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262e894-fb5a-4fb5-9d45-467310d816c5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:13.015598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442fd28-a8fb-4dc7-bff9-57979ed82255 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:13.016331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442fd28-a8fb-4dc7-bff9-57979ed82255 service placement] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:13.016827 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442fd28-a8fb-4dc7-bff9-57979ed82255 service placement] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:13.017134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442fd28-a8fb-4dc7-bff9-57979ed82255 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:13.019596 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2442fd28-a8fb-4dc7-bff9-57979ed82255 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:13.033807 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442fd28-a8fb-4dc7-bff9-57979ed82255 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:13.035505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442fd28-a8fb-4dc7-bff9-57979ed82255 service placement] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:13.047937 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1377/2704] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:09:13.604147 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b3abc-f68c-4859-97bc-e2180c4942cf service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:13.689426 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b3abc-f68c-4859-97bc-e2180c4942cf service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:13.694917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:13.696185 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:13.696635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:13.697148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] PATH_INFO: `/v3/users/ad7191a8ee2d4630b9bddff0403d7688` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:13.699298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7191a8ee2d4630b9bddff0403d7688)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:13.702308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:13.722799 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:13.933438 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7191a8ee2d4630b9bddff0403d7688'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:13.955989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7191a8ee2d4630b9bddff0403d7688'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:13.957849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7191a8ee2d4630b9bddff0403d7688'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:14.071545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7191a8ee2d4630b9bddff0403d7688'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:14.098485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7191a8ee2d4630b9bddff0403d7688'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:14.117566 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1328/2705] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:09:13 2022] DELETE /identity/v3/users/ad7191a8ee2d4630b9bddff0403d7688 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:14.123910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485dc1f-57a3-41c2-a082-bad2368e5ab8 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:14.124720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485dc1f-57a3-41c2-a082-bad2368e5ab8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:14.125424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485dc1f-57a3-41c2-a082-bad2368e5ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:14.161553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-1485dc1f-57a3-41c2-a082-bad2368e5ab8 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:14.164324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1485dc1f-57a3-41c2-a082-bad2368e5ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:14.185348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:14.260618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1378/2706] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:14.274026 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:14.277199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:14.385716 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23238abb-c7fc-487e-b1e2-5138f17b3e99 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:14.394840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a64fd-dbbd-4d0f-a13c-3f00218cbc07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:14.395978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a64fd-dbbd-4d0f-a13c-3f00218cbc07 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:14.396538 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a64fd-dbbd-4d0f-a13c-3f00218cbc07 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:14.397077 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a64fd-dbbd-4d0f-a13c-3f00218cbc07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:14.416227 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a64fd-dbbd-4d0f-a13c-3f00218cbc07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:14.419693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a64fd-dbbd-4d0f-a13c-3f00218cbc07 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:14.444360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1329/2707] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:14.987700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485dc1f-57a3-41c2-a082-bad2368e5ab8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:14.991295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1485dc1f-57a3-41c2-a082-bad2368e5ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:15.021529 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485dc1f-57a3-41c2-a082-bad2368e5ab8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:15.026815 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:15.028580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:15.029872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:15.029872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] PATH_INFO: `/v3/projects/31d4d37fc0ee42af995b8717d76810e8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:15.031210 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d4d37fc0ee42af995b8717d76810e8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:15.033516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:15.165252 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d4d37fc0ee42af995b8717d76810e8', 'request_id': 'req-ba24768d-961b-4fd8-926e-c321c6891478'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:15.171553 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d4d37fc0ee42af995b8717d76810e8', 'request_id': 'req-ba24768d-961b-4fd8-926e-c321c6891478'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:15.178207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d4d37fc0ee42af995b8717d76810e8', 'request_id': 'req-ba24768d-961b-4fd8-926e-c321c6891478'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:15.303196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] The token cache is being invalidate because project 31d4d37fc0ee42af995b8717d76810e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:09:15.303719 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:15.304783 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ba24768d-961b-4fd8-926e-c321c6891478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:15.306713 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1379/2708] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:09:14 2022] DELETE /identity/v3/projects/31d4d37fc0ee42af995b8717d76810e8 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:15.317608 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f0be-c160-4f7e-953e-7df2dcccb8b2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:15.318429 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f0be-c160-4f7e-953e-7df2dcccb8b2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:15.318604 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f0be-c160-4f7e-953e-7df2dcccb8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:15.340739 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6817f0be-c160-4f7e-953e-7df2dcccb8b2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:15.342092 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6817f0be-c160-4f7e-953e-7df2dcccb8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:15.423736 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1330/2709] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:15.434584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8daa5-722e-4253-8033-08b244dcb0ea None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:15.434909 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8daa5-722e-4253-8033-08b244dcb0ea None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:15.435191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8daa5-722e-4253-8033-08b244dcb0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:15.456995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-18a8daa5-722e-4253-8033-08b244dcb0ea None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:15.458362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-18a8daa5-722e-4253-8033-08b244dcb0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:15.466599 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1380/2710] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:15.471267 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f0be-c160-4f7e-953e-7df2dcccb8b2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:15.484309 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f0be-c160-4f7e-953e-7df2dcccb8b2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:15.489854 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d046cd3-e36c-4a39-ac9e-203875a79dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:15.490586 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d046cd3-e36c-4a39-ac9e-203875a79dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:15.490872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d046cd3-e36c-4a39-ac9e-203875a79dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:15.491209 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d046cd3-e36c-4a39-ac9e-203875a79dc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:15.491976 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d046cd3-e36c-4a39-ac9e-203875a79dc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:09:15.492283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d046cd3-e36c-4a39-ac9e-203875a79dc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:15.493962 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d046cd3-e36c-4a39-ac9e-203875a79dc7 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:15.509405 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1331/2711] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:09:15.522381 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8daa5-722e-4253-8033-08b244dcb0ea None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:15.535862 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8daa5-722e-4253-8033-08b244dcb0ea None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:15.540711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de23112-37e1-4a1a-baf4-9a02b27aea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:15.541387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de23112-37e1-4a1a-baf4-9a02b27aea4f admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:15.541671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de23112-37e1-4a1a-baf4-9a02b27aea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:15.541950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de23112-37e1-4a1a-baf4-9a02b27aea4f admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:15.542658 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de23112-37e1-4a1a-baf4-9a02b27aea4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:15.545169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de23112-37e1-4a1a-baf4-9a02b27aea4f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:15.745620 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1381/2712] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:09:15 2022] POST /identity/v3/projects => generated 373 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:09:15.750843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d046cd3-e36c-4a39-ac9e-203875a79dc7 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:15.761482 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d046cd3-e36c-4a39-ac9e-203875a79dc7 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:15.783358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:15.844677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb59ff9-5b22-485a-9561-2b5ed3efa289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:15.846133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb59ff9-5b22-485a-9561-2b5ed3efa289 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:15.846133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb59ff9-5b22-485a-9561-2b5ed3efa289 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:15.846133 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb59ff9-5b22-485a-9561-2b5ed3efa289 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:15.848416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb59ff9-5b22-485a-9561-2b5ed3efa289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:15.848704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb59ff9-5b22-485a-9561-2b5ed3efa289 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:15.979893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1332/2713] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:09:15 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:09:15.982703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de23112-37e1-4a1a-baf4-9a02b27aea4f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:16.001188 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de23112-37e1-4a1a-baf4-9a02b27aea4f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:16.009224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87939d-5c2a-47e1-9b71-9f0213f9e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:16.010241 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87939d-5c2a-47e1-9b71-9f0213f9e59a admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:16.010731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87939d-5c2a-47e1-9b71-9f0213f9e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:16.011194 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87939d-5c2a-47e1-9b71-9f0213f9e59a admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:16.012723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87939d-5c2a-47e1-9b71-9f0213f9e59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:16.015822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87939d-5c2a-47e1-9b71-9f0213f9e59a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:16.034088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1382/2714] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:09:16.038448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb59ff9-5b22-485a-9561-2b5ed3efa289 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:16.053358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb59ff9-5b22-485a-9561-2b5ed3efa289 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:16.058683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f76aa5-6b7a-42b3-937d-c29d073ae092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:16.061451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f76aa5-6b7a-42b3-937d-c29d073ae092 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:16.061451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f76aa5-6b7a-42b3-937d-c29d073ae092 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:16.061451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f76aa5-6b7a-42b3-937d-c29d073ae092 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:16.062323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f76aa5-6b7a-42b3-937d-c29d073ae092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:16.064718 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f76aa5-6b7a-42b3-937d-c29d073ae092 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:16.077243 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1333/2715] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:09:16.081603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87939d-5c2a-47e1-9b71-9f0213f9e59a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:16.092782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87939d-5c2a-47e1-9b71-9f0213f9e59a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:16.098507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3085775-7e7e-48b6-9585-8b7a77b7c74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:16.099526 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3085775-7e7e-48b6-9585-8b7a77b7c74e admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:16.099717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3085775-7e7e-48b6-9585-8b7a77b7c74e admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:16.099999 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3085775-7e7e-48b6-9585-8b7a77b7c74e admin admin] PATH_INFO: `/v3/projects/b07d4fc8657d4ec79ab7b32fd651ecfe/users/52f20540c26749d6a4681aa82120a1c4/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:16.117684 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3085775-7e7e-48b6-9585-8b7a77b7c74e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07d4fc8657d4ec79ab7b32fd651ecfe, user_id=52f20540c26749d6a4681aa82120a1c4, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:16.120351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3085775-7e7e-48b6-9585-8b7a77b7c74e admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:16.190772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1383/2716] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:09:16 2022] PUT /identity/v3/projects/b07d4fc8657d4ec79ab7b32fd651ecfe/users/52f20540c26749d6a4681aa82120a1c4/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:16.201728 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c71bdd-0e19-468b-9c22-aae97161d88d None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:16.202019 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c71bdd-0e19-468b-9c22-aae97161d88d None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:16.202305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c71bdd-0e19-468b-9c22-aae97161d88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:16.230967 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d1c71bdd-0e19-468b-9c22-aae97161d88d None None] MFA Rules not processed for user `52f20540c26749d6a4681aa82120a1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:16.232197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d1c71bdd-0e19-468b-9c22-aae97161d88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:16.303910 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1334/2717] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:16.315667 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3085775-7e7e-48b6-9585-8b7a77b7c74e admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:16.317642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a3085775-7e7e-48b6-9585-8b7a77b7c74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:16.362694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:16.378109 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3085775-7e7e-48b6-9585-8b7a77b7c74e admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:16.388977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e390e-ae2e-40c7-ae11-a18d478f12fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:16.388977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e390e-ae2e-40c7-ae11-a18d478f12fb service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:16.388977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e390e-ae2e-40c7-ae11-a18d478f12fb service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:16.388977 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e390e-ae2e-40c7-ae11-a18d478f12fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:16.395936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e390e-ae2e-40c7-ae11-a18d478f12fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:16.397936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e390e-ae2e-40c7-ae11-a18d478f12fb service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:16.409058 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1384/2718] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:16.429688 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c71bdd-0e19-468b-9c22-aae97161d88d None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:16.516022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c71bdd-0e19-468b-9c22-aae97161d88d None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:16.522465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:16.523654 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:16.524638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:16.524638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] PATH_INFO: `/v3/users/52f20540c26749d6a4681aa82120a1c4` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:16.526129 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] RBAC: Authorizing `identity:delete_user(user_id=52f20540c26749d6a4681aa82120a1c4)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:16.528551 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:16.768935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:16.781989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52f20540c26749d6a4681aa82120a1c4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:16.789041 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52f20540c26749d6a4681aa82120a1c4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:16.789760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52f20540c26749d6a4681aa82120a1c4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:16.917577 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52f20540c26749d6a4681aa82120a1c4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:16.935297 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52f20540c26749d6a4681aa82120a1c4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:16.944889 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1335/2719] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:09:16 2022] DELETE /identity/v3/users/52f20540c26749d6a4681aa82120a1c4 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:16.949442 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb50c0-46aa-4f11-acc1-bbdd71441c4b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:16.949771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb50c0-46aa-4f11-acc1-bbdd71441c4b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:16.950578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb50c0-46aa-4f11-acc1-bbdd71441c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:16.972372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2deb50c0-46aa-4f11-acc1-bbdd71441c4b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:16.976416 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2deb50c0-46aa-4f11-acc1-bbdd71441c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:17.054208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1385/2720] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:17.065081 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:17.066689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:17.146989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7602c-d011-4acf-9f4f-f3be885ef03f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:17.151986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cca7f6-3ad9-4a85-bc26-a6d1930470b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:17.152695 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cca7f6-3ad9-4a85-bc26-a6d1930470b9 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:17.152993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cca7f6-3ad9-4a85-bc26-a6d1930470b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:17.153273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cca7f6-3ad9-4a85-bc26-a6d1930470b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:17.164052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cca7f6-3ad9-4a85-bc26-a6d1930470b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:17.165550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cca7f6-3ad9-4a85-bc26-a6d1930470b9 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:17.176935 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1336/2721] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:17.930570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb50c0-46aa-4f11-acc1-bbdd71441c4b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:17.932149 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2deb50c0-46aa-4f11-acc1-bbdd71441c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:17.946782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb50c0-46aa-4f11-acc1-bbdd71441c4b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:17.953001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:17.954374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:17.954580 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:17.955044 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] PATH_INFO: `/v3/projects/b07d4fc8657d4ec79ab7b32fd651ecfe` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:17.956472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07d4fc8657d4ec79ab7b32fd651ecfe)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:17.958657 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:18.087572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07d4fc8657d4ec79ab7b32fd651ecfe', 'request_id': 'req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:18.096438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07d4fc8657d4ec79ab7b32fd651ecfe', 'request_id': 'req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:18.103904 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07d4fc8657d4ec79ab7b32fd651ecfe', 'request_id': 'req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:18.168512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] The token cache is being invalidate because project b07d4fc8657d4ec79ab7b32fd651ecfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:09:18.169001 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:18.169668 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-e1a67caa-b25a-4d39-a61c-eea1d6c7213b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:18.171246 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1386/2722] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:09:17 2022] DELETE /identity/v3/projects/b07d4fc8657d4ec79ab7b32fd651ecfe => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:18.176545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a56ad-88cb-4484-894e-4306d596c0c7 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:18.177110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a56ad-88cb-4484-894e-4306d596c0c7 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:18.177110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a56ad-88cb-4484-894e-4306d596c0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:18.199246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3c9a56ad-88cb-4484-894e-4306d596c0c7 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:18.201074 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3c9a56ad-88cb-4484-894e-4306d596c0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:18.272028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1337/2723] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:18.280617 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0585a-1d78-4c8e-9c2d-89a2f813e609 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:18.280931 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0585a-1d78-4c8e-9c2d-89a2f813e609 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:18.281771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0585a-1d78-4c8e-9c2d-89a2f813e609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:18.304006 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-c7f0585a-1d78-4c8e-9c2d-89a2f813e609 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:18.305319 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c7f0585a-1d78-4c8e-9c2d-89a2f813e609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:18.315707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1387/2724] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:09:18.320542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a56ad-88cb-4484-894e-4306d596c0c7 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:18.332080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a56ad-88cb-4484-894e-4306d596c0c7 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:18.337172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94ae19-4879-4b9a-8944-e4de58bc6ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:18.339825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94ae19-4879-4b9a-8944-e4de58bc6ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:18.339825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94ae19-4879-4b9a-8944-e4de58bc6ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:18.339825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94ae19-4879-4b9a-8944-e4de58bc6ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:18.339825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94ae19-4879-4b9a-8944-e4de58bc6ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:09:18.339825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94ae19-4879-4b9a-8944-e4de58bc6ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:18.341366 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94ae19-4879-4b9a-8944-e4de58bc6ce0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:18.355624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1338/2725] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:09:18.360786 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0585a-1d78-4c8e-9c2d-89a2f813e609 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:18.370934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0585a-1d78-4c8e-9c2d-89a2f813e609 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:18.376214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2ee2e-9173-4a11-a659-9e3ae7ab3a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:18.376933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2ee2e-9173-4a11-a659-9e3ae7ab3a02 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:18.377223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2ee2e-9173-4a11-a659-9e3ae7ab3a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:18.377504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2ee2e-9173-4a11-a659-9e3ae7ab3a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:18.378213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2ee2e-9173-4a11-a659-9e3ae7ab3a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:18.380147 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2ee2e-9173-4a11-a659-9e3ae7ab3a02 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:18.485126 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1388/2726] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:09:18 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:09:18.489700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94ae19-4879-4b9a-8944-e4de58bc6ce0 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:18.500722 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94ae19-4879-4b9a-8944-e4de58bc6ce0 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:18.541491 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:18.560931 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb58ede-aeb5-45d1-8531-7663422267a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:18.561871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb58ede-aeb5-45d1-8531-7663422267a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:18.562162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb58ede-aeb5-45d1-8531-7663422267a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:18.562444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb58ede-aeb5-45d1-8531-7663422267a2 admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:18.563170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb58ede-aeb5-45d1-8531-7663422267a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:18.565139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb58ede-aeb5-45d1-8531-7663422267a2 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:18.676137 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1339/2727] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:09:18 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:09:18.680167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2ee2e-9173-4a11-a659-9e3ae7ab3a02 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:18.691510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2ee2e-9173-4a11-a659-9e3ae7ab3a02 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:18.696054 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea16c6-9dae-4330-bf3b-43a7d55df9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:18.696741 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea16c6-9dae-4330-bf3b-43a7d55df9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:18.697032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea16c6-9dae-4330-bf3b-43a7d55df9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:18.697313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea16c6-9dae-4330-bf3b-43a7d55df9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:18.698043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea16c6-9dae-4330-bf3b-43a7d55df9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:18.699666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea16c6-9dae-4330-bf3b-43a7d55df9b0 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:18.710450 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1389/2728] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:09:18.714189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb58ede-aeb5-45d1-8531-7663422267a2 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:18.724376 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb58ede-aeb5-45d1-8531-7663422267a2 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:18.729596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77466a2d-3449-4751-ae26-00ad1d40bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:18.730316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77466a2d-3449-4751-ae26-00ad1d40bab4 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:18.730643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77466a2d-3449-4751-ae26-00ad1d40bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:18.731470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77466a2d-3449-4751-ae26-00ad1d40bab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:18.732264 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77466a2d-3449-4751-ae26-00ad1d40bab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:18.733980 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77466a2d-3449-4751-ae26-00ad1d40bab4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:18.746175 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1340/2729] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:09:18.751236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea16c6-9dae-4330-bf3b-43a7d55df9b0 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:18.761355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea16c6-9dae-4330-bf3b-43a7d55df9b0 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:18.765988 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f37c2-5f84-4ce0-bfd8-f60195a19444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:18.766881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f37c2-5f84-4ce0-bfd8-f60195a19444 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:18.767167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f37c2-5f84-4ce0-bfd8-f60195a19444 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:18.767447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f37c2-5f84-4ce0-bfd8-f60195a19444 admin admin] PATH_INFO: `/v3/projects/4b62892e1c2c454abdf03d59ee726418/users/f6f590a0a5094a3380db0473baf61b3a/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:18.783807 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f37c2-5f84-4ce0-bfd8-f60195a19444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b62892e1c2c454abdf03d59ee726418, user_id=f6f590a0a5094a3380db0473baf61b3a, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:18.787962 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f37c2-5f84-4ce0-bfd8-f60195a19444 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:18.896042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:18.900948 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1390/2730] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:09:18 2022] PUT /identity/v3/projects/4b62892e1c2c454abdf03d59ee726418/users/f6f590a0a5094a3380db0473baf61b3a/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:18.913352 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e82b2-1ac0-430a-beb7-d42b3c299de6 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:18.913638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e82b2-1ac0-430a-beb7-d42b3c299de6 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:18.913923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e82b2-1ac0-430a-beb7-d42b3c299de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:18.943949 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-cc9e82b2-1ac0-430a-beb7-d42b3c299de6 None None] MFA Rules not processed for user `f6f590a0a5094a3380db0473baf61b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:18.945117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-cc9e82b2-1ac0-430a-beb7-d42b3c299de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:19.018335 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1341/2731] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:19.029518 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f37c2-5f84-4ce0-bfd8-f60195a19444 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:19.031215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-af9f37c2-5f84-4ce0-bfd8-f60195a19444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:19.093849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f37c2-5f84-4ce0-bfd8-f60195a19444 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:19.097567 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f462f-4a08-49bb-88f0-90b86e5cfab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:19.098264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f462f-4a08-49bb-88f0-90b86e5cfab6 service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:19.098500 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f462f-4a08-49bb-88f0-90b86e5cfab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:19.098771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f462f-4a08-49bb-88f0-90b86e5cfab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:19.108434 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f462f-4a08-49bb-88f0-90b86e5cfab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:19.110043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f462f-4a08-49bb-88f0-90b86e5cfab6 service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:19.122672 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1391/2732] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:29.015139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:29.115334 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=26 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:29.586916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e82b2-1ac0-430a-beb7-d42b3c299de6 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:29.656734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e82b2-1ac0-430a-beb7-d42b3c299de6 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:29.661748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:29.662643 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:29.662932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:29.663215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] PATH_INFO: `/v3/users/f6f590a0a5094a3380db0473baf61b3a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:29.665017 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6f590a0a5094a3380db0473baf61b3a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:29.667061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:29.908676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6f590a0a5094a3380db0473baf61b3a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:29.919392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6f590a0a5094a3380db0473baf61b3a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:29.919392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6f590a0a5094a3380db0473baf61b3a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:29.995067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6f590a0a5094a3380db0473baf61b3a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:30.008714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6f590a0a5094a3380db0473baf61b3a'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:30.017787 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1342/2733] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:09:29 2022] DELETE /identity/v3/users/f6f590a0a5094a3380db0473baf61b3a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:30.023982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a757811-12b0-49a9-b65f-1e032d743a2e None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:30.024332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a757811-12b0-49a9-b65f-1e032d743a2e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:30.024707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a757811-12b0-49a9-b65f-1e032d743a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:30.047525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-0a757811-12b0-49a9-b65f-1e032d743a2e None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:30.048777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0a757811-12b0-49a9-b65f-1e032d743a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:30.115245 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1392/2734] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:30.125886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:30.127349 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:30.188217 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a105-edb5-4ade-bfa2-3c330095f91b admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:30.191959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11ba2d-a656-48c0-b5fe-0a77f6db1ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:30.192625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11ba2d-a656-48c0-b5fe-0a77f6db1ddc service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:30.192964 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11ba2d-a656-48c0-b5fe-0a77f6db1ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:30.193250 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11ba2d-a656-48c0-b5fe-0a77f6db1ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:30.203956 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11ba2d-a656-48c0-b5fe-0a77f6db1ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:30.205466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11ba2d-a656-48c0-b5fe-0a77f6db1ddc service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:30.216174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1343/2735] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:30.667571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a757811-12b0-49a9-b65f-1e032d743a2e None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:30.669114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0a757811-12b0-49a9-b65f-1e032d743a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:30.681242 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a757811-12b0-49a9-b65f-1e032d743a2e None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:30.687249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:30.688139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:30.688419 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:30.688729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] PATH_INFO: `/v3/projects/4b62892e1c2c454abdf03d59ee726418` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:30.689971 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b62892e1c2c454abdf03d59ee726418)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:30.691883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:30.810208 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b62892e1c2c454abdf03d59ee726418', 'request_id': 'req-9bb7074a-4de5-41e9-a069-5edfb1a415b9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:30.817191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b62892e1c2c454abdf03d59ee726418', 'request_id': 'req-9bb7074a-4de5-41e9-a069-5edfb1a415b9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:30.823563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b62892e1c2c454abdf03d59ee726418', 'request_id': 'req-9bb7074a-4de5-41e9-a069-5edfb1a415b9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:30.905720 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] The token cache is being invalidate because project 4b62892e1c2c454abdf03d59ee726418 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:09:30.905998 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:30.906825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9bb7074a-4de5-41e9-a069-5edfb1a415b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:09:30.908414 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1393/2736] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:09:30 2022] DELETE /identity/v3/projects/4b62892e1c2c454abdf03d59ee726418 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:30.915634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71650948-cd42-4fce-9625-780d5116b89f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:30.915634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71650948-cd42-4fce-9625-780d5116b89f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:30.915634 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71650948-cd42-4fce-9625-780d5116b89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:30.933346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-71650948-cd42-4fce-9625-780d5116b89f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:30.935170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-71650948-cd42-4fce-9625-780d5116b89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:30.953966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:31.001127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1344/2737] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:31.009311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7721556-34d4-486b-859e-b69b98f6c4d5 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:31.009813 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7721556-34d4-486b-859e-b69b98f6c4d5 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:31.010150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7721556-34d4-486b-859e-b69b98f6c4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:31.029550 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-a7721556-34d4-486b-859e-b69b98f6c4d5 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:31.030725 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a7721556-34d4-486b-859e-b69b98f6c4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:31.039692 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1394/2738] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:31.044506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71650948-cd42-4fce-9625-780d5116b89f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:31.055241 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71650948-cd42-4fce-9625-780d5116b89f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:31.060578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f33b1d-2869-440f-af51-93c8ad69bdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:31.061276 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f33b1d-2869-440f-af51-93c8ad69bdaa admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:31.061829 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f33b1d-2869-440f-af51-93c8ad69bdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:31.062110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f33b1d-2869-440f-af51-93c8ad69bdaa admin admin] PATH_INFO: `/v3/domains` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:31.062843 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f33b1d-2869-440f-af51-93c8ad69bdaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:09:31.063156 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f33b1d-2869-440f-af51-93c8ad69bdaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:31.064797 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f33b1d-2869-440f-af51-93c8ad69bdaa admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:31.076700 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1345/2739] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:09:31.081031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7721556-34d4-486b-859e-b69b98f6c4d5 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:31.090193 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7721556-34d4-486b-859e-b69b98f6c4d5 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:31.094887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf3168-e2e3-4eb5-8929-2e9512f949d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:31.095558 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf3168-e2e3-4eb5-8929-2e9512f949d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:31.095833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf3168-e2e3-4eb5-8929-2e9512f949d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:31.096116 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf3168-e2e3-4eb5-8929-2e9512f949d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:31.096854 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf3168-e2e3-4eb5-8929-2e9512f949d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:31.099770 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf3168-e2e3-4eb5-8929-2e9512f949d2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:31.181195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1395/2740] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:09:31 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:09:31.185850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f33b1d-2869-440f-af51-93c8ad69bdaa admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:31.196244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f33b1d-2869-440f-af51-93c8ad69bdaa admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:31.252628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5db0ec-56e5-4d59-add3-3f7c7ba61abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:31.253325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5db0ec-56e5-4d59-add3-3f7c7ba61abf admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:31.253618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5db0ec-56e5-4d59-add3-3f7c7ba61abf admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:31.253905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5db0ec-56e5-4d59-add3-3f7c7ba61abf admin admin] PATH_INFO: `/v3/users` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:31.254598 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5db0ec-56e5-4d59-add3-3f7c7ba61abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:31.256499 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5db0ec-56e5-4d59-add3-3f7c7ba61abf admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:31.354332 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1346/2741] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:09:31 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:09:31.356926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf3168-e2e3-4eb5-8929-2e9512f949d2 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:31.367537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf3168-e2e3-4eb5-8929-2e9512f949d2 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:31.372196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b2830-ba7a-41a1-a4d6-9d755be12dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:31.372926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b2830-ba7a-41a1-a4d6-9d755be12dee admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:31.373216 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b2830-ba7a-41a1-a4d6-9d755be12dee admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:31.373492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b2830-ba7a-41a1-a4d6-9d755be12dee admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:31.374218 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b2830-ba7a-41a1-a4d6-9d755be12dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:31.375873 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b2830-ba7a-41a1-a4d6-9d755be12dee admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:31.386437 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1396/2742] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:09:31.391275 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5db0ec-56e5-4d59-add3-3f7c7ba61abf admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:31.401448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5db0ec-56e5-4d59-add3-3f7c7ba61abf admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:31.406723 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f9546-936a-499a-aa1a-ebda7387826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:31.407397 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f9546-936a-499a-aa1a-ebda7387826b admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:31.407677 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f9546-936a-499a-aa1a-ebda7387826b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:31.407954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f9546-936a-499a-aa1a-ebda7387826b admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:31.408704 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f9546-936a-499a-aa1a-ebda7387826b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:31.410347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f9546-936a-499a-aa1a-ebda7387826b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:31.421916 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1347/2743] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:09:31.425943 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b2830-ba7a-41a1-a4d6-9d755be12dee admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:31.438841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b2830-ba7a-41a1-a4d6-9d755be12dee admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:31.440957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f2326-711c-41fd-8335-5e69264636e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:31.441834 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f2326-711c-41fd-8335-5e69264636e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:31.442116 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f2326-711c-41fd-8335-5e69264636e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:31.442387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f2326-711c-41fd-8335-5e69264636e6 admin admin] PATH_INFO: `/v3/projects/2cb694ea9cbf458489f9f67a0ff6c40a/users/38037205c86a46ba98336f635c0dfb4e/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:31.457096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f2326-711c-41fd-8335-5e69264636e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb694ea9cbf458489f9f67a0ff6c40a, user_id=38037205c86a46ba98336f635c0dfb4e, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:31.459758 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f2326-711c-41fd-8335-5e69264636e6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:31.543482 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1397/2744] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:09:31 2022] PUT /identity/v3/projects/2cb694ea9cbf458489f9f67a0ff6c40a/users/38037205c86a46ba98336f635c0dfb4e/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:09:31.549287 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff1862-160e-4c01-b411-2c78a2f2673a None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:31.549578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff1862-160e-4c01-b411-2c78a2f2673a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:31.549855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff1862-160e-4c01-b411-2c78a2f2673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:31.567987 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-1cff1862-160e-4c01-b411-2c78a2f2673a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:31.569240 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-1cff1862-160e-4c01-b411-2c78a2f2673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:31.628064 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1348/2745] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:31.638396 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f2326-711c-41fd-8335-5e69264636e6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:31.640101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0c4f2326-711c-41fd-8335-5e69264636e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:31.662207 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:31.697528 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f2326-711c-41fd-8335-5e69264636e6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:31.702872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cbb2c-4656-4d27-9adf-e8bb8c09641d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:31.705648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cbb2c-4656-4d27-9adf-e8bb8c09641d service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:31.705648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cbb2c-4656-4d27-9adf-e8bb8c09641d service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:31.705648 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cbb2c-4656-4d27-9adf-e8bb8c09641d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:31.723224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cbb2c-4656-4d27-9adf-e8bb8c09641d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:31.725103 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cbb2c-4656-4d27-9adf-e8bb8c09641d service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:31.738754 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1398/2746] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:32.992999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b61ea-418e-4e03-81af-7a52c87ef030 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:32.993539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b61ea-418e-4e03-81af-7a52c87ef030 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:32.993539 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b61ea-418e-4e03-81af-7a52c87ef030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:33.027007 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-286b61ea-418e-4e03-81af-7a52c87ef030 None None] MFA Rules not processed for user `38037205c86a46ba98336f635c0dfb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:33.028424 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-286b61ea-418e-4e03-81af-7a52c87ef030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:33.046984 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:33.099573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1349/2747] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:33.113003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cbb2c-4656-4d27-9adf-e8bb8c09641d service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:33.114478 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a98cbb2c-4656-4d27-9adf-e8bb8c09641d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:33.173809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cbb2c-4656-4d27-9adf-e8bb8c09641d service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:33.177355 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5542c14-549f-4b41-a622-4e14721c211b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:33.178004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5542c14-549f-4b41-a622-4e14721c211b service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:33.178291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5542c14-549f-4b41-a622-4e14721c211b service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:33.178572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5542c14-549f-4b41-a622-4e14721c211b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:33.187965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5542c14-549f-4b41-a622-4e14721c211b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:33.189422 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5542c14-549f-4b41-a622-4e14721c211b service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:33.198603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1399/2748] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:34.440057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b61ea-418e-4e03-81af-7a52c87ef030 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:09:34.456811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b61ea-418e-4e03-81af-7a52c87ef030 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:09:34.461897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21e1f-50a2-4849-876e-83f6a44dbe3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:09:34.461897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21e1f-50a2-4849-876e-83f6a44dbe3e service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:34.461897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21e1f-50a2-4849-876e-83f6a44dbe3e service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:34.461897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21e1f-50a2-4849-876e-83f6a44dbe3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:34.463776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a0e21e1f-50a2-4849-876e-83f6a44dbe3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:34.527042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e21e1f-50a2-4849-876e-83f6a44dbe3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:09:34.528495 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e21e1f-50a2-4849-876e-83f6a44dbe3e service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:09:34.538195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1350/2749] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:09:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:09:34.931793 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87652-65c2-4321-81fe-3b907b02ed6e None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:34.932335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87652-65c2-4321-81fe-3b907b02ed6e None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:34.932335 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87652-65c2-4321-81fe-3b907b02ed6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:34.949491 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f1e87652-65c2-4321-81fe-3b907b02ed6e None None] MFA Rules not processed for user `38037205c86a46ba98336f635c0dfb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:34.950687 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f1e87652-65c2-4321-81fe-3b907b02ed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:34.957613 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1400/2750] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:44.330726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3605ffb-f906-4d2e-bda5-b172d08914bc None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:09:44.331306 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3605ffb-f906-4d2e-bda5-b172d08914bc None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:09:44.331415 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3605ffb-f906-4d2e-bda5-b172d08914bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:09:44.352979 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-a3605ffb-f906-4d2e-bda5-b172d08914bc None None] MFA Rules not processed for user `7ed43701d36b4b479a2a3a978d242625`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:09:44.354354 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-a3605ffb-f906-4d2e-bda5-b172d08914bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:09:44.362378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1351/2751] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:09:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:09:44.944682 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=46 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:09:54.347865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=50 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:10:04.716602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e87652-65c2-4321-81fe-3b907b02ed6e None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:10:04.718151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f1e87652-65c2-4321-81fe-3b907b02ed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:10:04.737504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e87652-65c2-4321-81fe-3b907b02ed6e None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:10:04.741568 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab8a32-fddd-4a75-ac5c-30ac299ccd94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:10:04.742934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab8a32-fddd-4a75-ac5c-30ac299ccd94 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:10:04.742934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab8a32-fddd-4a75-ac5c-30ac299ccd94 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:10:04.743290 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab8a32-fddd-4a75-ac5c-30ac299ccd94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:10:04.745386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-60ab8a32-fddd-4a75-ac5c-30ac299ccd94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:10:04.813195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab8a32-fddd-4a75-ac5c-30ac299ccd94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:10:04.814652 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab8a32-fddd-4a75-ac5c-30ac299ccd94 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:10:04.824559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1401/2752] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:10:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:10:14.820024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=30 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:10:49.886278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577859c-2942-4f16-a05a-af827ce428a2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:10:49.886858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577859c-2942-4f16-a05a-af827ce428a2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:10:49.887302 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577859c-2942-4f16-a05a-af827ce428a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:10:49.915103 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-2577859c-2942-4f16-a05a-af827ce428a2 None None] MFA Rules not processed for user `38037205c86a46ba98336f635c0dfb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:10:49.916793 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2577859c-2942-4f16-a05a-af827ce428a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:10:49.930160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1352/2753] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:10:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:10:57.116250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab8a32-fddd-4a75-ac5c-30ac299ccd94 service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:10:57.145214 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab8a32-fddd-4a75-ac5c-30ac299ccd94 service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:10:57.149398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:10:57.150077 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:10:57.150369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:10:57.150686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:10:57.152627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:10:57.166671 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:10:57.168134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:10:57.179965 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1402/2754] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:10:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:10:59.908188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:11:01.457595 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577859c-2942-4f16-a05a-af827ce428a2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:11:01.460110 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2577859c-2942-4f16-a05a-af827ce428a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:01.554587 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577859c-2942-4f16-a05a-af827ce428a2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:11:01.559894 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2a6bb-b8b5-4d68-8c04-72f34e50c3e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:11:01.560574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2a6bb-b8b5-4d68-8c04-72f34e50c3e6 service placement] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:11:01.561428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2a6bb-b8b5-4d68-8c04-72f34e50c3e6 service placement] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:11:01.561428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2a6bb-b8b5-4d68-8c04-72f34e50c3e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:11:01.563549 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-01c2a6bb-b8b5-4d68-8c04-72f34e50c3e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:01.576565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2a6bb-b8b5-4d68-8c04-72f34e50c3e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:11:01.580832 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2a6bb-b8b5-4d68-8c04-72f34e50c3e6 service placement] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:11:01.594599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1353/2755] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:11:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:11:06.430768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:11:06.432202 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:06.446181 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735568af-2bc5-46df-8ad6-54f4ae779b6a service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:11:06.450808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:11:06.451683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:11:06.451971 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:11:06.452279 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] PATH_INFO: `/v3/users/427fa281fc374bd2b9cd7b7fe59297a3` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:11:06.453502 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=427fa281fc374bd2b9cd7b7fe59297a3)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:11:06.456973 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:11:06.649291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '427fa281fc374bd2b9cd7b7fe59297a3'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:06.663372 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '427fa281fc374bd2b9cd7b7fe59297a3'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:06.671063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '427fa281fc374bd2b9cd7b7fe59297a3'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:06.671830 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '427fa281fc374bd2b9cd7b7fe59297a3'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:06.772926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '427fa281fc374bd2b9cd7b7fe59297a3'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:06.779782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1403/2756] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:11:06 2022] DELETE /identity/v3/users/427fa281fc374bd2b9cd7b7fe59297a3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:11:06.853778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2a6bb-b8b5-4d68-8c04-72f34e50c3e6 service placement] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:11:06.864951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2a6bb-b8b5-4d68-8c04-72f34e50c3e6 service placement] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:11:06.967199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b006213-9c1f-4d1d-b5c2-1e41a5deabed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:11:06.968903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b006213-9c1f-4d1d-b5c2-1e41a5deabed service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:11:06.969517 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b006213-9c1f-4d1d-b5c2-1e41a5deabed service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:11:06.970095 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b006213-9c1f-4d1d-b5c2-1e41a5deabed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:11:06.978868 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-9b006213-9c1f-4d1d-b5c2-1e41a5deabed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:07.074296 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b006213-9c1f-4d1d-b5c2-1e41a5deabed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:11:07.075769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b006213-9c1f-4d1d-b5c2-1e41a5deabed service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:11:07.088835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1354/2757] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:11:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:11:07.127755 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:11:07.129504 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:07.238447 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b79792-824a-4067-90b8-a135ea5beb2e admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:11:07.243331 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:11:07.244289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:11:07.244666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:11:07.245032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] PATH_INFO: `/v3/projects/d4f1f24b3a384f1db3a6e782253814ec` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:11:07.246557 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f1f24b3a384f1db3a6e782253814ec)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:11:07.248457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:11:07.377088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f1f24b3a384f1db3a6e782253814ec', 'request_id': 'req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:07.389849 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f1f24b3a384f1db3a6e782253814ec', 'request_id': 'req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:07.402102 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:11:07.413910 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f1f24b3a384f1db3a6e782253814ec', 'request_id': 'req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:07.465903 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] The token cache is being invalidate because project d4f1f24b3a384f1db3a6e782253814ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:11:07.466950 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:07.467343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:07.469792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1404/2758] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:11:07 2022] DELETE /identity/v3/projects/d4f1f24b3a384f1db3a6e782253814ec => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:11:07.475531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b006213-9c1f-4d1d-b5c2-1e41a5deabed service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:11:07.487958 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b006213-9c1f-4d1d-b5c2-1e41a5deabed service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:11:07.524867 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=100 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:11:07.577263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:11:07.578245 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:11:07.578533 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:11:07.578814 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] PATH_INFO: `/v3/users/bd25fe6aed024fc6b75f1bf449ce7321` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:11:07.580054 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd25fe6aed024fc6b75f1bf449ce7321)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:11:07.582067 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:11:07.855792 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd25fe6aed024fc6b75f1bf449ce7321'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:07.871307 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd25fe6aed024fc6b75f1bf449ce7321'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:07.872179 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd25fe6aed024fc6b75f1bf449ce7321'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:08.024940 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd25fe6aed024fc6b75f1bf449ce7321'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:08.044284 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd25fe6aed024fc6b75f1bf449ce7321'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:08.052411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1355/2759] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:11:07 2022] DELETE /identity/v3/users/bd25fe6aed024fc6b75f1bf449ce7321 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:11:08.377493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:11:08.379554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:08.402039 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c8e45-f755-4df3-ba31-44f2f4925ab7 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:11:08.479623 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:11:08.480560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:11:08.480884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:11:08.481157 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] PATH_INFO: `/v3/projects/fd7606f6fdc54b33b7490691669a08a2` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:11:08.482435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7606f6fdc54b33b7490691669a08a2)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:11:08.484423 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:11:08.605061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7606f6fdc54b33b7490691669a08a2', 'request_id': 'req-b3878f78-2d01-4bcb-9821-56e83f0e24fc'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:08.612241 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7606f6fdc54b33b7490691669a08a2', 'request_id': 'req-b3878f78-2d01-4bcb-9821-56e83f0e24fc'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:08.619698 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7606f6fdc54b33b7490691669a08a2', 'request_id': 'req-b3878f78-2d01-4bcb-9821-56e83f0e24fc'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:08.715358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] The token cache is being invalidate because project fd7606f6fdc54b33b7490691669a08a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86807) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:11:08.715585 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:08.716191 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:11:08.717792 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1405/2760] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:11:08 2022] DELETE /identity/v3/projects/fd7606f6fdc54b33b7490691669a08a2 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:11:18.047202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:11:18.711951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=54,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:11:18.717567 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:11:18.719267 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:18.787257 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6ea57-69b9-4c8e-923c-3b553393ce93 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:11:18.840416 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:11:18.841428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:11:18.841855 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:11:18.842432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:11:18.844300 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:18.929128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:11:18.930522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:11:18.943835 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1356/2761] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:11:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:11:28.938061 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=74 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:11:43.631196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:11:43.635436 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:43.650282 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3878f78-2d01-4bcb-9821-56e83f0e24fc admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:11:43.657745 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:11:43.658424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:11:43.658706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:11:43.658982 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:11:43.661569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:43.677104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:11:43.679752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:11:43.690942 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1406/2762] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:11:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:11:46.421089 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:11:46.422702 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:46.438350 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d319c-27c9-4c17-8c57-9bbbde4cf1f1 service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:11:46.445078 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:11:46.445738 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:11:46.446022 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:11:46.446298 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:11:46.448116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:11:46.462239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:11:46.468389 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:11:46.476809 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1357/2763] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:11:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:11:53.685187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=6 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:11:56.472281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=6 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:12:01.112066 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:01.113636 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:01.123808 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caea9cf-a4f9-46fb-8239-7ef1e1b20fbd service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:01.127619 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:01.128848 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:01.129133 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:01.129408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:01.131259 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:01.202731 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:01.205309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:01.223844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1407/2764] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:12:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:12:02.801770 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:02.804220 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:02.899202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b4f80-415a-4eeb-b061-286b7a72cf62 service nova] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:02.905989 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee97cc-81f5-4b2c-9548-71bfed17085c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:02.906540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee97cc-81f5-4b2c-9548-71bfed17085c service placement] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:02.906840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee97cc-81f5-4b2c-9548-71bfed17085c service placement] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:02.907134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee97cc-81f5-4b2c-9548-71bfed17085c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:02.910223 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-bbee97cc-81f5-4b2c-9548-71bfed17085c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:02.926628 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee97cc-81f5-4b2c-9548-71bfed17085c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:02.928912 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee97cc-81f5-4b2c-9548-71bfed17085c service placement] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:02.943028 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1358/2765] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:12:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 24 20:12:03.002860 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:03.004516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:03.026506 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d95b37-22ef-4d19-9744-6702399ee25d service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:03.031729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:03.032814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:03.033185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:03.033488 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] PATH_INFO: `/v3/users/38037205c86a46ba98336f635c0dfb4e` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:03.035570 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38037205c86a46ba98336f635c0dfb4e)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:03.037631 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:03.343057 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38037205c86a46ba98336f635c0dfb4e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:03.373689 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38037205c86a46ba98336f635c0dfb4e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:03.390180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38037205c86a46ba98336f635c0dfb4e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:03.391621 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38037205c86a46ba98336f635c0dfb4e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:03.567896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38037205c86a46ba98336f635c0dfb4e'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:03.583348 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1408/2766] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:12:03 2022] DELETE /identity/v3/users/38037205c86a46ba98336f635c0dfb4e => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:12:03.851282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee97cc-81f5-4b2c-9548-71bfed17085c service placement] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:03.865428 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee97cc-81f5-4b2c-9548-71bfed17085c service placement] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:03.937809 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:03.938719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:03.938999 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:03.939535 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] PATH_INFO: `/v3/projects/2cb694ea9cbf458489f9f67a0ff6c40a` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:03.941357 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb694ea9cbf458489f9f67a0ff6c40a)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:03.943356 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:04.072679 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb694ea9cbf458489f9f67a0ff6c40a', 'request_id': 'req-06021912-f806-43ad-917a-cfee1492eb4f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:04.079953 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb694ea9cbf458489f9f67a0ff6c40a', 'request_id': 'req-06021912-f806-43ad-917a-cfee1492eb4f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:04.161594 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb694ea9cbf458489f9f67a0ff6c40a', 'request_id': 'req-06021912-f806-43ad-917a-cfee1492eb4f'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:04.179388 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] The token cache is being invalidate because project 2cb694ea9cbf458489f9f67a0ff6c40a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:12:04.180097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:04.180887 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:04.182707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1359/2767] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:12:03 2022] DELETE /identity/v3/projects/2cb694ea9cbf458489f9f67a0ff6c40a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:12:10.508839 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:10.510359 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:10.547095 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d958049-7860-4fb2-bbb8-cb94472234b6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:10.615169 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:10.617017 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:10.617531 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:10.618013 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] PATH_INFO: `/v3/users/7ed43701d36b4b479a2a3a978d242625` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:10.620135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed43701d36b4b479a2a3a978d242625)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:10.623386 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:10.632569 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:12:10.862869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed43701d36b4b479a2a3a978d242625'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:10.876941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed43701d36b4b479a2a3a978d242625'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:10.885474 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed43701d36b4b479a2a3a978d242625'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:10.886457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed43701d36b4b479a2a3a978d242625'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:11.006395 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed43701d36b4b479a2a3a978d242625'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:11.014197 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1409/2768] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:12:10 2022] DELETE /identity/v3/users/7ed43701d36b4b479a2a3a978d242625 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:12:11.402216 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:11.405339 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:11.426893 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06021912-f806-43ad-917a-cfee1492eb4f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:11.459343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:12:11.522154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:11.523810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:11.523810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:11.523810 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] PATH_INFO: `/v3/projects/757b69afcbc64d9a853a7ae1f7dc3307` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:11.525048 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=757b69afcbc64d9a853a7ae1f7dc3307)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:11.527534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:11.688097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757b69afcbc64d9a853a7ae1f7dc3307', 'request_id': 'req-b8f49434-ca84-4a92-abdf-cf9b8613a791'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:11.696410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757b69afcbc64d9a853a7ae1f7dc3307', 'request_id': 'req-b8f49434-ca84-4a92-abdf-cf9b8613a791'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:11.773755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757b69afcbc64d9a853a7ae1f7dc3307', 'request_id': 'req-b8f49434-ca84-4a92-abdf-cf9b8613a791'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:11.805115 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] The token cache is being invalidate because project 757b69afcbc64d9a853a7ae1f7dc3307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:12:11.805398 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:11.806181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:11.808102 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1360/2769] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:12:11 2022] DELETE /identity/v3/projects/757b69afcbc64d9a853a7ae1f7dc3307 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:12:11.813088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:11.817517 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:11.835351 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdc559-d7b6-4085-a1d3-cfdf57018918 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:11.910311 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:11.911268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:11.911554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:11.911832 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] PATH_INFO: `/v3/users/fb20ceed686b468bbff462e7272b29cb` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:11.913222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb20ceed686b468bbff462e7272b29cb)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:11.915233 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:12.113844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb20ceed686b468bbff462e7272b29cb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.133639 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb20ceed686b468bbff462e7272b29cb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.145632 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb20ceed686b468bbff462e7272b29cb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.146991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb20ceed686b468bbff462e7272b29cb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.211771 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-00cf56c6-a45f-4d66-8960-c18ff2f09500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb20ceed686b468bbff462e7272b29cb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.220079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1410/2770] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:12:11 2022] DELETE /identity/v3/users/fb20ceed686b468bbff462e7272b29cb => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:12:12.516254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:12.519615 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:12.546049 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f49434-ca84-4a92-abdf-cf9b8613a791 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:12.648794 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:12.649735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:12.649993 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:12.650280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] PATH_INFO: `/v3/projects/ffb48bb456c8458483f20ab4e2a0462c` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:12.651472 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb48bb456c8458483f20ab4e2a0462c)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:12.653420 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:12.802681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb48bb456c8458483f20ab4e2a0462c', 'request_id': 'req-8722b6e4-ed08-4618-9336-6ff09c3e8618'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.804961 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:12:12.811966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb48bb456c8458483f20ab4e2a0462c', 'request_id': 'req-8722b6e4-ed08-4618-9336-6ff09c3e8618'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.937456 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb48bb456c8458483f20ab4e2a0462c', 'request_id': 'req-8722b6e4-ed08-4618-9336-6ff09c3e8618'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.950070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] The token cache is being invalidate because project ffb48bb456c8458483f20ab4e2a0462c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:12:12.950362 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.951047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:12:12.952873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1361/2771] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:12:12 2022] DELETE /identity/v3/projects/ffb48bb456c8458483f20ab4e2a0462c => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:12:15.133734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4570e-12e0-4c26-95b5-9ac531685933 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:15.134248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4570e-12e0-4c26-95b5-9ac531685933 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:15.134371 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4570e-12e0-4c26-95b5-9ac531685933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:15.141345 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:12:15.152379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-dfa4570e-12e0-4c26-95b5-9ac531685933 None None] MFA Rules not processed for user `59e631f2d387466aa2ce6ab2be0414a0`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:12:15.153710 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-dfa4570e-12e0-4c26-95b5-9ac531685933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:15.215602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1411/2772] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:12:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:12:15.524348 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:15.525959 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:15.591680 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722b6e4-ed08-4618-9336-6ff09c3e8618 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:15.597043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:15.597707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:15.597994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:15.598268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:15.600086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:15.610448 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:15.611901 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:15.621403 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1362/2773] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:12:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:12:16.040114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4570e-12e0-4c26-95b5-9ac531685933 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:16.041766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-dfa4570e-12e0-4c26-95b5-9ac531685933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:16.122451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4570e-12e0-4c26-95b5-9ac531685933 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:16.128364 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:16.129067 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:16.129354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:16.129683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:16.131545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:16.146308 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:16.147974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:16.161375 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1412/2774] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:12:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:12:24.545097 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:12:24.546783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:24.645521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1ff42-cc87-458c-85cb-cae343bc302b service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:12:24.651346 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3eb23-1ef6-46a3-8435-bd4a8c2c0635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123b7f659c6a4ceba18ce268b1335542', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:12:24.652034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3eb23-1ef6-46a3-8435-bd4a8c2c0635 service nova] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:12:24.652323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3eb23-1ef6-46a3-8435-bd4a8c2c0635 service nova] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:12:24.652611 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3eb23-1ef6-46a3-8435-bd4a8c2c0635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:12:24.654596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-70b3eb23-1ef6-46a3-8435-bd4a8c2c0635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:12:24.673405 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3eb23-1ef6-46a3-8435-bd4a8c2c0635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:12:24.675683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3eb23-1ef6-46a3-8435-bd4a8c2c0635 service nova] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:12:24.689380 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1363/2775] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:12:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:12:26.154642 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=55 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:12:34.682985 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=65,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:41.086564 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:41.088110 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:41.162563 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49e6b9-21d5-4c01-800d-946bfbe56dee service placement] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:41.167304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:41.168186 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:41.168473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:41.168785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] PATH_INFO: `/v3/users/59e631f2d387466aa2ce6ab2be0414a0` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:41.169981 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59e631f2d387466aa2ce6ab2be0414a0)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:41.172087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:41.355660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59e631f2d387466aa2ce6ab2be0414a0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.367856 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59e631f2d387466aa2ce6ab2be0414a0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.375061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59e631f2d387466aa2ce6ab2be0414a0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.375768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59e631f2d387466aa2ce6ab2be0414a0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.438281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b3b9ef08-a06b-4fb4-8e5d-2fec2824edf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59e631f2d387466aa2ce6ab2be0414a0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.445159 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1413/2776] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:13:41 2022] DELETE /identity/v3/users/59e631f2d387466aa2ce6ab2be0414a0 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:13:41.685534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3eb23-1ef6-46a3-8435-bd4a8c2c0635 service nova] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:41.697602 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3eb23-1ef6-46a3-8435-bd4a8c2c0635 service nova] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:41.758280 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:41.759188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:41.759478 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:41.759758 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] PATH_INFO: `/v3/projects/8fac69862bde46c8b40d8a4cc26ca379` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:41.760998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fac69862bde46c8b40d8a4cc26ca379)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:41.762928 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:41.879883 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fac69862bde46c8b40d8a4cc26ca379', 'request_id': 'req-c8005282-e73d-4adc-a8af-d4a9db61c2f1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.886705 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fac69862bde46c8b40d8a4cc26ca379', 'request_id': 'req-c8005282-e73d-4adc-a8af-d4a9db61c2f1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.975681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fac69862bde46c8b40d8a4cc26ca379', 'request_id': 'req-c8005282-e73d-4adc-a8af-d4a9db61c2f1'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.986882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] The token cache is being invalidate because project 8fac69862bde46c8b40d8a4cc26ca379 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:13:41.987172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.987801 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c8005282-e73d-4adc-a8af-d4a9db61c2f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:41.989393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1364/2777] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:13:41 2022] DELETE /identity/v3/projects/8fac69862bde46c8b40d8a4cc26ca379 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:41.994800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec7323-d075-4250-a3cf-c2f4f84c9505 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:41.995092 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec7323-d075-4250-a3cf-c2f4f84c9505 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:41.995373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec7323-d075-4250-a3cf-c2f4f84c9505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.013917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-4bec7323-d075-4250-a3cf-c2f4f84c9505 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:42.015117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4bec7323-d075-4250-a3cf-c2f4f84c9505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:42.072842 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1414/2778] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:13:42.080849 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddfcd1-6553-4f61-91a4-97fb2f83c395 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:42.081146 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddfcd1-6553-4f61-91a4-97fb2f83c395 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:42.085895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddfcd1-6553-4f61-91a4-97fb2f83c395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.099865 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-b4ddfcd1-6553-4f61-91a4-97fb2f83c395 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:42.101151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b4ddfcd1-6553-4f61-91a4-97fb2f83c395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:42.109412 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1365/2779] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:13:42.113591 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec7323-d075-4250-a3cf-c2f4f84c9505 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:42.124240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec7323-d075-4250-a3cf-c2f4f84c9505 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:42.128918 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59f67d-f46a-4f95-869e-7b036f38a57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:42.129581 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59f67d-f46a-4f95-869e-7b036f38a57f admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:42.129865 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59f67d-f46a-4f95-869e-7b036f38a57f admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:42.130146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59f67d-f46a-4f95-869e-7b036f38a57f admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.130875 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59f67d-f46a-4f95-869e-7b036f38a57f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:13:42.131190 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59f67d-f46a-4f95-869e-7b036f38a57f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:42.133451 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59f67d-f46a-4f95-869e-7b036f38a57f admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:42.145032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1415/2780] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:13:42.149360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddfcd1-6553-4f61-91a4-97fb2f83c395 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:42.158946 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddfcd1-6553-4f61-91a4-97fb2f83c395 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:42.164208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4e84e-adaa-44ac-a2e1-033642a8468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:42.164897 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4e84e-adaa-44ac-a2e1-033642a8468a admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:42.165189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4e84e-adaa-44ac-a2e1-033642a8468a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:42.165469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4e84e-adaa-44ac-a2e1-033642a8468a admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.166254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4e84e-adaa-44ac-a2e1-033642a8468a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:42.168060 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4e84e-adaa-44ac-a2e1-033642a8468a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:42.240202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1366/2781] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:13:42 2022] POST /identity/v3/projects => generated 403 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:42.245085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59f67d-f46a-4f95-869e-7b036f38a57f admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:42.255035 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59f67d-f46a-4f95-869e-7b036f38a57f admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:42.258598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:42.311298 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671ca0e-6c47-4ed0-82ed-3a6058fe5310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:42.311974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671ca0e-6c47-4ed0-82ed-3a6058fe5310 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:42.312264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671ca0e-6c47-4ed0-82ed-3a6058fe5310 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:42.312545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671ca0e-6c47-4ed0-82ed-3a6058fe5310 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.313304 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671ca0e-6c47-4ed0-82ed-3a6058fe5310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:42.315196 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671ca0e-6c47-4ed0-82ed-3a6058fe5310 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:42.391711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1416/2782] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:13:42 2022] POST /identity/v3/users => generated 302 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:42.395937 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4e84e-adaa-44ac-a2e1-033642a8468a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:42.407308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4e84e-adaa-44ac-a2e1-033642a8468a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:42.412181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013967f-c5c1-4e84-b54e-2882a296075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:42.412884 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013967f-c5c1-4e84-b54e-2882a296075c admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:42.413195 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013967f-c5c1-4e84-b54e-2882a296075c admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:42.413476 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013967f-c5c1-4e84-b54e-2882a296075c admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.414206 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013967f-c5c1-4e84-b54e-2882a296075c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:42.415804 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013967f-c5c1-4e84-b54e-2882a296075c admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:42.427104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1367/2783] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:42.431303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671ca0e-6c47-4ed0-82ed-3a6058fe5310 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:42.441469 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671ca0e-6c47-4ed0-82ed-3a6058fe5310 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:42.446696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d8a49-52b2-4277-8994-552568f70a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:42.447272 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d8a49-52b2-4277-8994-552568f70a45 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:42.447559 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d8a49-52b2-4277-8994-552568f70a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:42.447841 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d8a49-52b2-4277-8994-552568f70a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.448565 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d8a49-52b2-4277-8994-552568f70a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:42.450192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d8a49-52b2-4277-8994-552568f70a45 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:42.460038 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1417/2784] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:42.464877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013967f-c5c1-4e84-b54e-2882a296075c admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:42.474841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013967f-c5c1-4e84-b54e-2882a296075c admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:42.479446 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40844c0b-2a31-4386-b921-a6ed40985d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:42.480331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40844c0b-2a31-4386-b921-a6ed40985d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:42.480616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40844c0b-2a31-4386-b921-a6ed40985d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:42.480926 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40844c0b-2a31-4386-b921-a6ed40985d81 admin admin] PATH_INFO: `/v3/projects/d859b94c5e4649d89d751503f997e85b/users/f7a178ce9e6043848125f596dadf893c/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.495618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40844c0b-2a31-4386-b921-a6ed40985d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d859b94c5e4649d89d751503f997e85b, user_id=f7a178ce9e6043848125f596dadf893c, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:42.498246 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40844c0b-2a31-4386-b921-a6ed40985d81 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:42.570278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1368/2785] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:13:42 2022] PUT /identity/v3/projects/d859b94c5e4649d89d751503f997e85b/users/f7a178ce9e6043848125f596dadf893c/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:42.575536 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb66df-5c40-404f-b461-33a2bb8b2f84 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:42.575848 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb66df-5c40-404f-b461-33a2bb8b2f84 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:42.576117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb66df-5c40-404f-b461-33a2bb8b2f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.593814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-11cb66df-5c40-404f-b461-33a2bb8b2f84 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:42.595022 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-11cb66df-5c40-404f-b461-33a2bb8b2f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:42.652472 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1418/2786] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:13:42.661891 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40844c0b-2a31-4386-b921-a6ed40985d81 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:42.663735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-40844c0b-2a31-4386-b921-a6ed40985d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:42.697617 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:42.724439 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40844c0b-2a31-4386-b921-a6ed40985d81 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:42.728534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d496c-ad48-4672-abc5-2aae303dd05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:42.729406 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d496c-ad48-4672-abc5-2aae303dd05a service neutron] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:42.730116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d496c-ad48-4672-abc5-2aae303dd05a service neutron] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:42.730616 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d496c-ad48-4672-abc5-2aae303dd05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:42.740618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d496c-ad48-4672-abc5-2aae303dd05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:42.742697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d496c-ad48-4672-abc5-2aae303dd05a service neutron] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:42.753187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1369/2787] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:13:44.039890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7daa7-3f6f-4b7f-b6b8-25b903088098 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:44.040490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7daa7-3f6f-4b7f-b6b8-25b903088098 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:44.040490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7daa7-3f6f-4b7f-b6b8-25b903088098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:44.068917 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-dfb7daa7-3f6f-4b7f-b6b8-25b903088098 None None] MFA Rules not processed for user `f7a178ce9e6043848125f596dadf893c`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:44.070866 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-dfb7daa7-3f6f-4b7f-b6b8-25b903088098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:44.142516 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1419/2788] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:13:44.147082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d496c-ad48-4672-abc5-2aae303dd05a service neutron] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:44.157218 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d496c-ad48-4672-abc5-2aae303dd05a service neutron] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:44.162030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848656c0-3bcc-4fcb-a015-1ea30ffd4abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:44.162707 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848656c0-3bcc-4fcb-a015-1ea30ffd4abb admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:44.162998 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848656c0-3bcc-4fcb-a015-1ea30ffd4abb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:44.163281 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848656c0-3bcc-4fcb-a015-1ea30ffd4abb admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:44.164023 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848656c0-3bcc-4fcb-a015-1ea30ffd4abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:44.165979 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848656c0-3bcc-4fcb-a015-1ea30ffd4abb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:44.286507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1370/2789] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:13:44 2022] POST /identity/v3/projects => generated 403 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:44.290525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7daa7-3f6f-4b7f-b6b8-25b903088098 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:44.301084 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7daa7-3f6f-4b7f-b6b8-25b903088098 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:44.313485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:44.358701 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c6c95-37e3-4702-8bd6-93e8cf15dbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:44.359544 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c6c95-37e3-4702-8bd6-93e8cf15dbfc admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:44.359711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c6c95-37e3-4702-8bd6-93e8cf15dbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:44.359933 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c6c95-37e3-4702-8bd6-93e8cf15dbfc admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:44.360665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c6c95-37e3-4702-8bd6-93e8cf15dbfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:44.362615 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c6c95-37e3-4702-8bd6-93e8cf15dbfc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:44.502320 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1420/2790] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:13:44 2022] POST /identity/v3/users => generated 308 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:44.506347 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848656c0-3bcc-4fcb-a015-1ea30ffd4abb admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:44.516744 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848656c0-3bcc-4fcb-a015-1ea30ffd4abb admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:44.521247 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398f60-d253-4cc6-b955-e11f4753f102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:44.521905 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398f60-d253-4cc6-b955-e11f4753f102 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:44.522188 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398f60-d253-4cc6-b955-e11f4753f102 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:44.522464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398f60-d253-4cc6-b955-e11f4753f102 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:44.523227 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398f60-d253-4cc6-b955-e11f4753f102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:44.524885 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398f60-d253-4cc6-b955-e11f4753f102 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:44.535729 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1371/2791] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:44.540480 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c6c95-37e3-4702-8bd6-93e8cf15dbfc admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:44.550809 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c6c95-37e3-4702-8bd6-93e8cf15dbfc admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:44.556199 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c33e6-65b6-4388-979d-d30c282dc015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:44.557135 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c33e6-65b6-4388-979d-d30c282dc015 admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:44.557424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c33e6-65b6-4388-979d-d30c282dc015 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:44.557777 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c33e6-65b6-4388-979d-d30c282dc015 admin admin] PATH_INFO: `/v3/projects/c0d61e9dda314f3ba84b708c6f756d6b/users/97f7ef64e75c481aa68a759ba690f981/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:44.573061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c33e6-65b6-4388-979d-d30c282dc015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d61e9dda314f3ba84b708c6f756d6b, user_id=97f7ef64e75c481aa68a759ba690f981, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:44.575723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c33e6-65b6-4388-979d-d30c282dc015 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:44.691463 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1421/2792] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:13:44 2022] PUT /identity/v3/projects/c0d61e9dda314f3ba84b708c6f756d6b/users/97f7ef64e75c481aa68a759ba690f981/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:44.696176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398f60-d253-4cc6-b955-e11f4753f102 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:44.707540 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398f60-d253-4cc6-b955-e11f4753f102 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:44.774043 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e438002-a940-4a29-8d07-8291b8196cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:44.774786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e438002-a940-4a29-8d07-8291b8196cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:44.775073 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e438002-a940-4a29-8d07-8291b8196cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:44.775351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e438002-a940-4a29-8d07-8291b8196cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:44.776091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e438002-a940-4a29-8d07-8291b8196cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:44.777748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e438002-a940-4a29-8d07-8291b8196cd0 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:44.788181 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1372/2793] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:44.793051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c33e6-65b6-4388-979d-d30c282dc015 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:44.803681 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c33e6-65b6-4388-979d-d30c282dc015 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:44.809105 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ef3cf-b8f7-464e-9fde-bba69bc078bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:44.809316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ef3cf-b8f7-464e-9fde-bba69bc078bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:44.809605 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ef3cf-b8f7-464e-9fde-bba69bc078bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:44.809887 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ef3cf-b8f7-464e-9fde-bba69bc078bd admin admin] PATH_INFO: `/v3/projects/c0d61e9dda314f3ba84b708c6f756d6b/users/97f7ef64e75c481aa68a759ba690f981/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:44.811881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ef3cf-b8f7-464e-9fde-bba69bc078bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d61e9dda314f3ba84b708c6f756d6b, user_id=97f7ef64e75c481aa68a759ba690f981, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:44.814609 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ef3cf-b8f7-464e-9fde-bba69bc078bd admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:44.825342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1422/2794] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:13:44 2022] PUT /identity/v3/projects/c0d61e9dda314f3ba84b708c6f756d6b/users/97f7ef64e75c481aa68a759ba690f981/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:45.977664 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6123745-ed1f-4699-adfe-0736f660a6ae None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:45.983336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6123745-ed1f-4699-adfe-0736f660a6ae None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:45.983336 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6123745-ed1f-4699-adfe-0736f660a6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:46.008726 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-d6123745-ed1f-4699-adfe-0736f660a6ae None None] MFA Rules not processed for user `97f7ef64e75c481aa68a759ba690f981`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:46.010774 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-d6123745-ed1f-4699-adfe-0736f660a6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:46.097786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1373/2795] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:13:47.416379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ef3cf-b8f7-464e-9fde-bba69bc078bd admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:47.482508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ef3cf-b8f7-464e-9fde-bba69bc078bd admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:47.487176 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:47.488046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:47.488337 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:47.489683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] PATH_INFO: `/v3/users/f7a178ce9e6043848125f596dadf893c` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:47.489840 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7a178ce9e6043848125f596dadf893c)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:47.491737 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:47.681028 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7a178ce9e6043848125f596dadf893c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:47.693140 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7a178ce9e6043848125f596dadf893c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:47.697532 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:47.700328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7a178ce9e6043848125f596dadf893c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:47.701043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7a178ce9e6043848125f596dadf893c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:47.787607 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7a178ce9e6043848125f596dadf893c'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:47.794926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1423/2796] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:13:47 2022] DELETE /identity/v3/users/f7a178ce9e6043848125f596dadf893c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:48.034150 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6123745-ed1f-4699-adfe-0736f660a6ae None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:48.045328 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6123745-ed1f-4699-adfe-0736f660a6ae None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:48.067024 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:48.109162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:48.110031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:48.110319 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:48.110599 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] PATH_INFO: `/v3/projects/d859b94c5e4649d89d751503f997e85b` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:48.118683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d859b94c5e4649d89d751503f997e85b)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:48.118683 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:48.222811 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd859b94c5e4649d89d751503f997e85b', 'request_id': 'req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.230173 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd859b94c5e4649d89d751503f997e85b', 'request_id': 'req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.290920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd859b94c5e4649d89d751503f997e85b', 'request_id': 'req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.302215 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] The token cache is being invalidate because project d859b94c5e4649d89d751503f997e85b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:13:48.302503 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.303152 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.305370 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1374/2797] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:13:48 2022] DELETE /identity/v3/projects/d859b94c5e4649d89d751503f997e85b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:48.309476 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:48.310899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:48.325087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208361ee-fb05-444c-8623-dfe90081c3c2 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:48.380578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:48.381477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:48.381759 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:48.382032 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] PATH_INFO: `/v3/users/97f7ef64e75c481aa68a759ba690f981` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:48.383281 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f7ef64e75c481aa68a759ba690f981)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:48.385215 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:48.515089 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f7ef64e75c481aa68a759ba690f981'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.527493 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f7ef64e75c481aa68a759ba690f981'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.534505 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f7ef64e75c481aa68a759ba690f981'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.535187 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f7ef64e75c481aa68a759ba690f981'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.581699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-cb38f9e1-03ff-47ae-ab6f-8d2461b57cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f7ef64e75c481aa68a759ba690f981'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:48.588547 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1424/2798] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:13:48 2022] DELETE /identity/v3/users/97f7ef64e75c481aa68a759ba690f981 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:48.841100 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:48.842525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:48.858511 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8009ae7-6732-4d1a-a38e-eb11e8af5bd2 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:48.918492 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:48.919358 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:48.919642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:48.919915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] PATH_INFO: `/v3/projects/c0d61e9dda314f3ba84b708c6f756d6b` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:48.921163 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0d61e9dda314f3ba84b708c6f756d6b)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:48.923052 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:49.079681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0d61e9dda314f3ba84b708c6f756d6b', 'request_id': 'req-1b6e6e80-b837-4f3a-a628-ab82886877a4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:49.080805 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:49.087558 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0d61e9dda314f3ba84b708c6f756d6b', 'request_id': 'req-1b6e6e80-b837-4f3a-a628-ab82886877a4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:49.175815 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0d61e9dda314f3ba84b708c6f756d6b', 'request_id': 'req-1b6e6e80-b837-4f3a-a628-ab82886877a4'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:49.187494 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] The token cache is being invalidate because project c0d61e9dda314f3ba84b708c6f756d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:13:49.190917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:49.190917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1b6e6e80-b837-4f3a-a628-ab82886877a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:49.190917 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1375/2799] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:13:48 2022] DELETE /identity/v3/projects/c0d61e9dda314f3ba84b708c6f756d6b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:49.196302 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32d5a1-a9fe-4423-90fb-73f3b93fb6b0 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.196590 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32d5a1-a9fe-4423-90fb-73f3b93fb6b0 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.196907 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32d5a1-a9fe-4423-90fb-73f3b93fb6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.215877 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-df32d5a1-a9fe-4423-90fb-73f3b93fb6b0 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:49.219379 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-df32d5a1-a9fe-4423-90fb-73f3b93fb6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:49.279161 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1425/2800] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:13:49.287189 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99331-20f6-4ed3-bb2f-014eb38259df None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.287484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99331-20f6-4ed3-bb2f-014eb38259df None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.287761 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99331-20f6-4ed3-bb2f-014eb38259df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.306681 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-ffc99331-20f6-4ed3-bb2f-014eb38259df None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:49.307871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ffc99331-20f6-4ed3-bb2f-014eb38259df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:49.316090 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1376/2801] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:13:49.321078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32d5a1-a9fe-4423-90fb-73f3b93fb6b0 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:49.330906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32d5a1-a9fe-4423-90fb-73f3b93fb6b0 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:49.335670 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523d688-c095-4d3c-a646-e7f7e3f613bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:49.336383 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523d688-c095-4d3c-a646-e7f7e3f613bb admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.336699 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523d688-c095-4d3c-a646-e7f7e3f613bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.336986 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523d688-c095-4d3c-a646-e7f7e3f613bb admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.337781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523d688-c095-4d3c-a646-e7f7e3f613bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:13:49.338097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523d688-c095-4d3c-a646-e7f7e3f613bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:49.339705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523d688-c095-4d3c-a646-e7f7e3f613bb admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:49.341485 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:49.352529 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1426/2802] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:13:49.361286 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc99331-20f6-4ed3-bb2f-014eb38259df None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:49.371735 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc99331-20f6-4ed3-bb2f-014eb38259df None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:49.379986 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3da01c-7b7f-4151-8121-92c2e5fb3fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:49.380780 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3da01c-7b7f-4151-8121-92c2e5fb3fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.381098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3da01c-7b7f-4151-8121-92c2e5fb3fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.381381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3da01c-7b7f-4151-8121-92c2e5fb3fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.382091 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3da01c-7b7f-4151-8121-92c2e5fb3fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:49.387331 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3da01c-7b7f-4151-8121-92c2e5fb3fd4 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:49.457841 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1377/2803] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:13:49 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:49.462273 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523d688-c095-4d3c-a646-e7f7e3f613bb admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:49.474262 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523d688-c095-4d3c-a646-e7f7e3f613bb admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:49.532263 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e40571-da6b-4b73-9fcf-3afd4d472f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:49.532976 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e40571-da6b-4b73-9fcf-3afd4d472f65 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.533260 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e40571-da6b-4b73-9fcf-3afd4d472f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.533539 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e40571-da6b-4b73-9fcf-3afd4d472f65 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.534236 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e40571-da6b-4b73-9fcf-3afd4d472f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:49.536154 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e40571-da6b-4b73-9fcf-3afd4d472f65 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:49.604884 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1427/2804] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:13:49 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:49.609128 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3da01c-7b7f-4151-8121-92c2e5fb3fd4 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:49.619860 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3da01c-7b7f-4151-8121-92c2e5fb3fd4 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:49.624779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2353a-3e56-43d1-be7e-964ffe8b9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:49.625449 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2353a-3e56-43d1-be7e-964ffe8b9604 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.625737 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2353a-3e56-43d1-be7e-964ffe8b9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.626075 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2353a-3e56-43d1-be7e-964ffe8b9604 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.626819 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2353a-3e56-43d1-be7e-964ffe8b9604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:49.628463 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2353a-3e56-43d1-be7e-964ffe8b9604 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:49.639828 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1378/2805] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:49.643766 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e40571-da6b-4b73-9fcf-3afd4d472f65 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:49.653627 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e40571-da6b-4b73-9fcf-3afd4d472f65 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:49.658288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e144ac-2801-4a3f-83ba-974dc2a5d2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:49.658939 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e144ac-2801-4a3f-83ba-974dc2a5d2cf admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.659228 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e144ac-2801-4a3f-83ba-974dc2a5d2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.659503 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e144ac-2801-4a3f-83ba-974dc2a5d2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.660220 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e144ac-2801-4a3f-83ba-974dc2a5d2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:49.661838 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e144ac-2801-4a3f-83ba-974dc2a5d2cf admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:49.672042 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1428/2806] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:49.676020 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2353a-3e56-43d1-be7e-964ffe8b9604 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:49.686532 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2353a-3e56-43d1-be7e-964ffe8b9604 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:49.691292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edb408-c9b1-4330-b925-e4b6c453713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:49.692165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edb408-c9b1-4330-b925-e4b6c453713a admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.692451 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edb408-c9b1-4330-b925-e4b6c453713a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.692763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edb408-c9b1-4330-b925-e4b6c453713a admin admin] PATH_INFO: `/v3/projects/37be79a0f3de4d4d96a1328a6db48981/users/d98afefa4bb640ee9d0fc88b816cfd63/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.707693 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edb408-c9b1-4330-b925-e4b6c453713a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37be79a0f3de4d4d96a1328a6db48981, user_id=d98afefa4bb640ee9d0fc88b816cfd63, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:49.710460 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edb408-c9b1-4330-b925-e4b6c453713a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:49.777069 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1379/2807] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:13:49 2022] PUT /identity/v3/projects/37be79a0f3de4d4d96a1328a6db48981/users/d98afefa4bb640ee9d0fc88b816cfd63/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:49.789602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0db5f-0ee8-4fa3-8a19-59f2cfe0b18c None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.789936 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0db5f-0ee8-4fa3-8a19-59f2cfe0b18c None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.790339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0db5f-0ee8-4fa3-8a19-59f2cfe0b18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.818872 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-bea0db5f-0ee8-4fa3-8a19-59f2cfe0b18c None None] MFA Rules not processed for user `d98afefa4bb640ee9d0fc88b816cfd63`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:49.820146 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-bea0db5f-0ee8-4fa3-8a19-59f2cfe0b18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:49.883836 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1429/2808] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:13:49.888368 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edb408-c9b1-4330-b925-e4b6c453713a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:49.949145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edb408-c9b1-4330-b925-e4b6c453713a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:49.953752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfefa6-bcc4-40fd-9322-02ab7e6fde75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:49.954411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfefa6-bcc4-40fd-9322-02ab7e6fde75 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:49.954697 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfefa6-bcc4-40fd-9322-02ab7e6fde75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:49.954975 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfefa6-bcc4-40fd-9322-02ab7e6fde75 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:49.955663 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfefa6-bcc4-40fd-9322-02ab7e6fde75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:49.957638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfefa6-bcc4-40fd-9322-02ab7e6fde75 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:50.034569 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1380/2809] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:13:49 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:50.039250 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0db5f-0ee8-4fa3-8a19-59f2cfe0b18c None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:50.050756 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0db5f-0ee8-4fa3-8a19-59f2cfe0b18c None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:50.110339 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85e2af-60a2-4e28-a513-54eb4d95cff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:50.111014 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85e2af-60a2-4e28-a513-54eb4d95cff7 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:50.111295 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85e2af-60a2-4e28-a513-54eb4d95cff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:50.111571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85e2af-60a2-4e28-a513-54eb4d95cff7 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:50.112267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85e2af-60a2-4e28-a513-54eb4d95cff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:50.114226 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85e2af-60a2-4e28-a513-54eb4d95cff7 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:50.122435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:50.222203 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1430/2810] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:13:50 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:50.225752 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfefa6-bcc4-40fd-9322-02ab7e6fde75 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:50.236458 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfefa6-bcc4-40fd-9322-02ab7e6fde75 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:50.241221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b856ace-dd93-455d-8268-119d00ff6b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:50.242475 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b856ace-dd93-455d-8268-119d00ff6b5e admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:50.242768 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b856ace-dd93-455d-8268-119d00ff6b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:50.243469 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b856ace-dd93-455d-8268-119d00ff6b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:50.243762 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b856ace-dd93-455d-8268-119d00ff6b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:50.245423 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b856ace-dd93-455d-8268-119d00ff6b5e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:50.256526 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1381/2811] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:50.262011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85e2af-60a2-4e28-a513-54eb4d95cff7 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:50.272705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85e2af-60a2-4e28-a513-54eb4d95cff7 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:50.277523 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8196bc9-4b53-4c86-923d-209438ce3ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:50.278192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8196bc9-4b53-4c86-923d-209438ce3ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:50.278583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8196bc9-4b53-4c86-923d-209438ce3ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:50.278867 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8196bc9-4b53-4c86-923d-209438ce3ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:50.279598 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8196bc9-4b53-4c86-923d-209438ce3ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:50.281258 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8196bc9-4b53-4c86-923d-209438ce3ee5 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:50.291325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1431/2812] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:50.295464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b856ace-dd93-455d-8268-119d00ff6b5e admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:50.306122 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b856ace-dd93-455d-8268-119d00ff6b5e admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:50.310720 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3de042-7638-458f-ab11-00e1adb03f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:50.311600 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3de042-7638-458f-ab11-00e1adb03f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:50.311944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3de042-7638-458f-ab11-00e1adb03f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:50.312162 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3de042-7638-458f-ab11-00e1adb03f55 admin admin] PATH_INFO: `/v3/projects/f171fabb1f57434bb8ba7048b1ff2462/users/7149bdf7bf99478d984aa698b7febed0/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:50.327490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3de042-7638-458f-ab11-00e1adb03f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f171fabb1f57434bb8ba7048b1ff2462, user_id=7149bdf7bf99478d984aa698b7febed0, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:50.330171 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3de042-7638-458f-ab11-00e1adb03f55 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:50.405009 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1382/2813] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:13:50 2022] PUT /identity/v3/projects/f171fabb1f57434bb8ba7048b1ff2462/users/7149bdf7bf99478d984aa698b7febed0/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:50.418222 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c4332-b918-4cd1-97e0-221bdf1a95e9 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:50.418512 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c4332-b918-4cd1-97e0-221bdf1a95e9 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:50.418800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c4332-b918-4cd1-97e0-221bdf1a95e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:50.450881 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-4c8c4332-b918-4cd1-97e0-221bdf1a95e9 None None] MFA Rules not processed for user `7149bdf7bf99478d984aa698b7febed0`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:50.453996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-4c8c4332-b918-4cd1-97e0-221bdf1a95e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:50.515703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1432/2814] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:13:50.520132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3de042-7638-458f-ab11-00e1adb03f55 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:50.547507 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:50.584353 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3de042-7638-458f-ab11-00e1adb03f55 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:50.589170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb2c8d-73e9-4850-8563-c3af008ee26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:50.589834 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb2c8d-73e9-4850-8563-c3af008ee26e admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:50.590117 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb2c8d-73e9-4850-8563-c3af008ee26e admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:50.590393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb2c8d-73e9-4850-8563-c3af008ee26e admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:50.591132 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb2c8d-73e9-4850-8563-c3af008ee26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:50.593153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb2c8d-73e9-4850-8563-c3af008ee26e admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:50.675107 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1383/2815] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:13:50 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:50.679063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c4332-b918-4cd1-97e0-221bdf1a95e9 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:50.691253 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c4332-b918-4cd1-97e0-221bdf1a95e9 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:50.751124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d49c33-ee54-40e2-8342-df10f12b6fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:50.751803 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d49c33-ee54-40e2-8342-df10f12b6fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:50.752091 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d49c33-ee54-40e2-8342-df10f12b6fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:50.752370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d49c33-ee54-40e2-8342-df10f12b6fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:50.753117 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d49c33-ee54-40e2-8342-df10f12b6fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:50.755063 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d49c33-ee54-40e2-8342-df10f12b6fa0 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:50.878878 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1433/2816] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:13:50 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:13:50.882896 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb2c8d-73e9-4850-8563-c3af008ee26e admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:50.893199 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb2c8d-73e9-4850-8563-c3af008ee26e admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:50.897741 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3038e-f73d-4c30-821a-508ca8454a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:50.898944 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3038e-f73d-4c30-821a-508ca8454a27 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:50.899229 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3038e-f73d-4c30-821a-508ca8454a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:50.899504 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3038e-f73d-4c30-821a-508ca8454a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:50.900272 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3038e-f73d-4c30-821a-508ca8454a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:50.901908 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3038e-f73d-4c30-821a-508ca8454a27 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:50.912665 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1384/2817] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:50.917462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d49c33-ee54-40e2-8342-df10f12b6fa0 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:50.927388 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d49c33-ee54-40e2-8342-df10f12b6fa0 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:50.931890 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89145f13-d4d9-4e81-94d0-53fb20e48dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:50.932800 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89145f13-d4d9-4e81-94d0-53fb20e48dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:50.933085 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89145f13-d4d9-4e81-94d0-53fb20e48dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:50.933362 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89145f13-d4d9-4e81-94d0-53fb20e48dcc admin admin] PATH_INFO: `/v3/projects/5a8ee8f072bc4e38a4fe1dcc7cd6ada1/users/a8e29e6f6d5244f19a7a82b08c5eebff/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:50.948151 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89145f13-d4d9-4e81-94d0-53fb20e48dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8ee8f072bc4e38a4fe1dcc7cd6ada1, user_id=a8e29e6f6d5244f19a7a82b08c5eebff, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:50.950820 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89145f13-d4d9-4e81-94d0-53fb20e48dcc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:51.027573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1434/2818] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:13:50 2022] PUT /identity/v3/projects/5a8ee8f072bc4e38a4fe1dcc7cd6ada1/users/a8e29e6f6d5244f19a7a82b08c5eebff/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:51.031554 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3038e-f73d-4c30-821a-508ca8454a27 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:51.042344 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3038e-f73d-4c30-821a-508ca8454a27 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:51.102522 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5cf5f-9aa1-4bb8-b753-5caef0a0ed0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:51.103222 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5cf5f-9aa1-4bb8-b753-5caef0a0ed0b admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:51.103513 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5cf5f-9aa1-4bb8-b753-5caef0a0ed0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:51.103792 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5cf5f-9aa1-4bb8-b753-5caef0a0ed0b admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:51.104521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5cf5f-9aa1-4bb8-b753-5caef0a0ed0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:51.106151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5cf5f-9aa1-4bb8-b753-5caef0a0ed0b admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:51.115840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1385/2819] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:51.120571 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89145f13-d4d9-4e81-94d0-53fb20e48dcc admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:51.131341 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89145f13-d4d9-4e81-94d0-53fb20e48dcc admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:51.136584 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540331b-b0d1-43cd-a7ae-ae0888816b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:51.137492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540331b-b0d1-43cd-a7ae-ae0888816b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:51.137773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540331b-b0d1-43cd-a7ae-ae0888816b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:51.138046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540331b-b0d1-43cd-a7ae-ae0888816b7a admin admin] PATH_INFO: `/v3/projects/5a8ee8f072bc4e38a4fe1dcc7cd6ada1/users/a8e29e6f6d5244f19a7a82b08c5eebff/roles/c688b8fdcef8429f85cb90f675f24ca8` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:51.140055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540331b-b0d1-43cd-a7ae-ae0888816b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8ee8f072bc4e38a4fe1dcc7cd6ada1, user_id=a8e29e6f6d5244f19a7a82b08c5eebff, role_id=c688b8fdcef8429f85cb90f675f24ca8)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:51.142675 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540331b-b0d1-43cd-a7ae-ae0888816b7a admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:51.153096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1435/2820] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:13:51 2022] PUT /identity/v3/projects/5a8ee8f072bc4e38a4fe1dcc7cd6ada1/users/a8e29e6f6d5244f19a7a82b08c5eebff/roles/c688b8fdcef8429f85cb90f675f24ca8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:51.164871 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36f681-461e-4f63-a7b7-22f02b1af14b None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:51.165170 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36f681-461e-4f63-a7b7-22f02b1af14b None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:51.165659 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36f681-461e-4f63-a7b7-22f02b1af14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:51.193689 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-4d36f681-461e-4f63-a7b7-22f02b1af14b None None] MFA Rules not processed for user `a8e29e6f6d5244f19a7a82b08c5eebff`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:51.194890 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4d36f681-461e-4f63-a7b7-22f02b1af14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:51.261877 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1386/2821] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:13:51.273231 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540331b-b0d1-43cd-a7ae-ae0888816b7a admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:51.274940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-a540331b-b0d1-43cd-a7ae-ae0888816b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:51.301401 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:51.332926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540331b-b0d1-43cd-a7ae-ae0888816b7a admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:51.337167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbf561-efd8-4dd0-bd51-01d2d8e184fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:51.337822 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbf561-efd8-4dd0-bd51-01d2d8e184fd service cinder] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:51.338112 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbf561-efd8-4dd0-bd51-01d2d8e184fd service cinder] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:51.338394 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbf561-efd8-4dd0-bd51-01d2d8e184fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:51.347454 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbf561-efd8-4dd0-bd51-01d2d8e184fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:51.348941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbf561-efd8-4dd0-bd51-01d2d8e184fd service cinder] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:51.359043 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1436/2822] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:13:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:13:51.448018 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36f681-461e-4f63-a7b7-22f02b1af14b None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:51.449468 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-4d36f681-461e-4f63-a7b7-22f02b1af14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:51.461772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36f681-461e-4f63-a7b7-22f02b1af14b None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:51.466371 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8207da-2397-4c5d-b401-0aa3802eca5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:51.467025 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8207da-2397-4c5d-b401-0aa3802eca5f service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:51.467320 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8207da-2397-4c5d-b401-0aa3802eca5f service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:51.467601 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8207da-2397-4c5d-b401-0aa3802eca5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:51.486068 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:51.527626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8207da-2397-4c5d-b401-0aa3802eca5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:51.529470 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8207da-2397-4c5d-b401-0aa3802eca5f service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:51.538490 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1387/2823] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:13:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:13:55.687442 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbf561-efd8-4dd0-bd51-01d2d8e184fd service cinder] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:55.750717 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbf561-efd8-4dd0-bd51-01d2d8e184fd service cinder] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:55.755249 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:55.757255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:55.757255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:55.757255 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] PATH_INFO: `/v3/users/d98afefa4bb640ee9d0fc88b816cfd63` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:55.757940 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98afefa4bb640ee9d0fc88b816cfd63)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:55.760387 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:55.939046 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98afefa4bb640ee9d0fc88b816cfd63'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:55.951467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98afefa4bb640ee9d0fc88b816cfd63'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:55.958237 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98afefa4bb640ee9d0fc88b816cfd63'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:55.958957 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98afefa4bb640ee9d0fc88b816cfd63'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:56.044310 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98afefa4bb640ee9d0fc88b816cfd63'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:56.050703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1437/2824] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:13:55 2022] DELETE /identity/v3/users/d98afefa4bb640ee9d0fc88b816cfd63 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:56.057476 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008206d4-18d5-4259-90b3-9d5c96fe691a None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:56.057476 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008206d4-18d5-4259-90b3-9d5c96fe691a None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:56.057476 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008206d4-18d5-4259-90b3-9d5c96fe691a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:56.074200 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-008206d4-18d5-4259-90b3-9d5c96fe691a None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:56.075464 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-008206d4-18d5-4259-90b3-9d5c96fe691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:56.135919 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1388/2825] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:13:56.145941 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:56.147417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:56.195365 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:56.207403 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfa057-20fb-4fc9-8c15-d208b2d43bce admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:56.211264 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a8443-99e0-42a5-a127-1602107a747c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:56.211934 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a8443-99e0-42a5-a127-1602107a747c service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:56.212224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a8443-99e0-42a5-a127-1602107a747c service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:56.212554 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a8443-99e0-42a5-a127-1602107a747c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:56.222128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a8443-99e0-42a5-a127-1602107a747c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:56.223734 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a8443-99e0-42a5-a127-1602107a747c service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:56.233061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1438/2826] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:13:56.610858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008206d4-18d5-4259-90b3-9d5c96fe691a None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:56.612372 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-008206d4-18d5-4259-90b3-9d5c96fe691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:56.625244 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008206d4-18d5-4259-90b3-9d5c96fe691a None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:56.629951 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:56.631050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:56.633851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:56.633851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] PATH_INFO: `/v3/projects/37be79a0f3de4d4d96a1328a6db48981` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:56.633851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37be79a0f3de4d4d96a1328a6db48981)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:56.634790 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:56.746593 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37be79a0f3de4d4d96a1328a6db48981', 'request_id': 'req-49c6b6f7-1f29-44b9-97a9-a0c069feef51'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:56.752874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37be79a0f3de4d4d96a1328a6db48981', 'request_id': 'req-49c6b6f7-1f29-44b9-97a9-a0c069feef51'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:56.816143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37be79a0f3de4d4d96a1328a6db48981', 'request_id': 'req-49c6b6f7-1f29-44b9-97a9-a0c069feef51'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:56.828222 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] The token cache is being invalidate because project 37be79a0f3de4d4d96a1328a6db48981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:13:56.830717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:56.830717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:56.830717 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1389/2827] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:13:56 2022] DELETE /identity/v3/projects/37be79a0f3de4d4d96a1328a6db48981 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:56.834804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a8443-99e0-42a5-a127-1602107a747c service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:56.845240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a8443-99e0-42a5-a127-1602107a747c service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:56.914533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:56.915424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:56.915711 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:56.915991 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] PATH_INFO: `/v3/users/7149bdf7bf99478d984aa698b7febed0` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:56.917833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7149bdf7bf99478d984aa698b7febed0)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:56.919768 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:57.076297 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7149bdf7bf99478d984aa698b7febed0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.088101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7149bdf7bf99478d984aa698b7febed0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.095691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7149bdf7bf99478d984aa698b7febed0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.096464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7149bdf7bf99478d984aa698b7febed0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.175736 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7149bdf7bf99478d984aa698b7febed0'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.182467 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1439/2828] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:13:56 2022] DELETE /identity/v3/users/7149bdf7bf99478d984aa698b7febed0 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:57.600165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:57.605138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:57.618030 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6b6f7-1f29-44b9-97a9-a0c069feef51 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:57.658580 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:57.675058 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:57.675932 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:57.676219 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:57.676496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] PATH_INFO: `/v3/projects/f171fabb1f57434bb8ba7048b1ff2462` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:57.677721 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f171fabb1f57434bb8ba7048b1ff2462)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:57.679633 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:57.806882 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f171fabb1f57434bb8ba7048b1ff2462', 'request_id': 'req-c96774ae-2b17-4110-868a-ed83eb70b826'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.813574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f171fabb1f57434bb8ba7048b1ff2462', 'request_id': 'req-c96774ae-2b17-4110-868a-ed83eb70b826'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.889148 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f171fabb1f57434bb8ba7048b1ff2462', 'request_id': 'req-c96774ae-2b17-4110-868a-ed83eb70b826'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.900239 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] The token cache is being invalidate because project f171fabb1f57434bb8ba7048b1ff2462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:13:57.903776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.903776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:57.903776 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1390/2829] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:13:57 2022] DELETE /identity/v3/projects/f171fabb1f57434bb8ba7048b1ff2462 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:57.907618 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:57.909051 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:57.921752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b84bee-1eff-4976-bf42-f0ed53c2f19e admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:57.977473 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:57.978354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:57.978644 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:57.978923 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] PATH_INFO: `/v3/users/a8e29e6f6d5244f19a7a82b08c5eebff` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:57.980115 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8e29e6f6d5244f19a7a82b08c5eebff)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:57.982079 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:57.987723 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:58.251464 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8e29e6f6d5244f19a7a82b08c5eebff'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:58.262968 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8e29e6f6d5244f19a7a82b08c5eebff'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:58.270610 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8e29e6f6d5244f19a7a82b08c5eebff'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:58.271330 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8e29e6f6d5244f19a7a82b08c5eebff'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:58.375829 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-b7bde2ee-a849-40cc-8f32-f9275330ce17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8e29e6f6d5244f19a7a82b08c5eebff'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:58.382328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1440/2830] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:13:57 2022] DELETE /identity/v3/users/a8e29e6f6d5244f19a7a82b08c5eebff => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:13:59.006626 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:59.008149 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:59.021506 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96774ae-2b17-4110-868a-ed83eb70b826 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:59.086283 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:59.087154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.087444 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.087724 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] PATH_INFO: `/v3/projects/5a8ee8f072bc4e38a4fe1dcc7cd6ada1` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.088963 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8ee8f072bc4e38a4fe1dcc7cd6ada1)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:59.090872 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:59.218214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8ee8f072bc4e38a4fe1dcc7cd6ada1', 'request_id': 'req-45242b8c-3c19-4a29-97a4-bc3e6409e148'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:59.224898 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8ee8f072bc4e38a4fe1dcc7cd6ada1', 'request_id': 'req-45242b8c-3c19-4a29-97a4-bc3e6409e148'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:59.290278 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8ee8f072bc4e38a4fe1dcc7cd6ada1', 'request_id': 'req-45242b8c-3c19-4a29-97a4-bc3e6409e148'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:59.301547 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] The token cache is being invalidate because project 5a8ee8f072bc4e38a4fe1dcc7cd6ada1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:13:59.301840 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:59.302484 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-45242b8c-3c19-4a29-97a4-bc3e6409e148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:13:59.304037 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1391/2831] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:13:59 2022] DELETE /identity/v3/projects/5a8ee8f072bc4e38a4fe1dcc7cd6ada1 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:13:59.311221 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0983fc-7e6e-4776-bbcb-226b29a8cdba None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.319560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0983fc-7e6e-4776-bbcb-226b29a8cdba None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.319560 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0983fc-7e6e-4776-bbcb-226b29a8cdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.330522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ef0983fc-7e6e-4776-bbcb-226b29a8cdba None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:59.331706 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ef0983fc-7e6e-4776-bbcb-226b29a8cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:59.391217 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1441/2832] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:13:59.399521 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f85905-9f9a-4205-aa64-428d11710e81 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.399815 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f85905-9f9a-4205-aa64-428d11710e81 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.400098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f85905-9f9a-4205-aa64-428d11710e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.407746 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:59.420208 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-78f85905-9f9a-4205-aa64-428d11710e81 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:59.421457 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-78f85905-9f9a-4205-aa64-428d11710e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:13:59.429568 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1392/2833] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:13:59.433814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0983fc-7e6e-4776-bbcb-226b29a8cdba None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:59.446683 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0983fc-7e6e-4776-bbcb-226b29a8cdba None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:59.454781 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4812d-f4e9-4fd8-b823-1b6790d02781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:59.455601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4812d-f4e9-4fd8-b823-1b6790d02781 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.455895 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4812d-f4e9-4fd8-b823-1b6790d02781 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.456180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4812d-f4e9-4fd8-b823-1b6790d02781 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.456971 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4812d-f4e9-4fd8-b823-1b6790d02781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:13:59.457288 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4812d-f4e9-4fd8-b823-1b6790d02781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:59.458896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4812d-f4e9-4fd8-b823-1b6790d02781 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:59.471913 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1442/2834] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:13:59.476355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f85905-9f9a-4205-aa64-428d11710e81 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:59.485833 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f85905-9f9a-4205-aa64-428d11710e81 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:59.490591 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36451e1-0644-4d7f-ba7c-37d6c94cc7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:59.491254 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36451e1-0644-4d7f-ba7c-37d6c94cc7fd admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.491545 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36451e1-0644-4d7f-ba7c-37d6c94cc7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.491825 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36451e1-0644-4d7f-ba7c-37d6c94cc7fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.492529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36451e1-0644-4d7f-ba7c-37d6c94cc7fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:59.494473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36451e1-0644-4d7f-ba7c-37d6c94cc7fd admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:59.566085 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1393/2835] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:13:59 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:59.570238 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4812d-f4e9-4fd8-b823-1b6790d02781 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:59.580928 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4812d-f4e9-4fd8-b823-1b6790d02781 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:59.636765 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a803e61-36e6-45d0-879c-410eb9e539de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:59.637462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803e61-36e6-45d0-879c-410eb9e539de admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.637747 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803e61-36e6-45d0-879c-410eb9e539de admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.638024 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803e61-36e6-45d0-879c-410eb9e539de admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.639358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a803e61-36e6-45d0-879c-410eb9e539de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:59.641312 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a803e61-36e6-45d0-879c-410eb9e539de admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:59.742040 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1443/2836] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:13:59 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:13:59.746232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36451e1-0644-4d7f-ba7c-37d6c94cc7fd admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:59.757404 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36451e1-0644-4d7f-ba7c-37d6c94cc7fd admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:59.762160 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d886c-b204-44da-9a1e-1fef108db334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:59.762853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d886c-b204-44da-9a1e-1fef108db334 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.763779 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d886c-b204-44da-9a1e-1fef108db334 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.764066 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d886c-b204-44da-9a1e-1fef108db334 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.764850 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d886c-b204-44da-9a1e-1fef108db334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:59.766463 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d886c-b204-44da-9a1e-1fef108db334 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:59.777076 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1394/2837] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:13:59.781705 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a803e61-36e6-45d0-879c-410eb9e539de admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:59.787050 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:13:59.792449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a803e61-36e6-45d0-879c-410eb9e539de admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:59.798373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc616-a64b-4bf5-a391-7a428c93e7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:59.798373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc616-a64b-4bf5-a391-7a428c93e7ad admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.798373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc616-a64b-4bf5-a391-7a428c93e7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.798703 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc616-a64b-4bf5-a391-7a428c93e7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.799309 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc616-a64b-4bf5-a391-7a428c93e7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:59.800925 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc616-a64b-4bf5-a391-7a428c93e7ad admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:59.812005 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1444/2838] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:13:59.816606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d886c-b204-44da-9a1e-1fef108db334 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:13:59.833557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d886c-b204-44da-9a1e-1fef108db334 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:13:59.838359 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235a7ed-ee41-44aa-9cff-b02bf4ec5814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:13:59.839252 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235a7ed-ee41-44aa-9cff-b02bf4ec5814 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.839537 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235a7ed-ee41-44aa-9cff-b02bf4ec5814 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.839813 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235a7ed-ee41-44aa-9cff-b02bf4ec5814 admin admin] PATH_INFO: `/v3/projects/1969f5fd958e4f9ab887aa944f91028c/users/7519ef837c7848158ee0b12e663ddab9/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.857588 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235a7ed-ee41-44aa-9cff-b02bf4ec5814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1969f5fd958e4f9ab887aa944f91028c, user_id=7519ef837c7848158ee0b12e663ddab9, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:13:59.860271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235a7ed-ee41-44aa-9cff-b02bf4ec5814 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:13:59.937202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1395/2839] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:13:59 2022] PUT /identity/v3/projects/1969f5fd958e4f9ab887aa944f91028c/users/7519ef837c7848158ee0b12e663ddab9/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:13:59.960404 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e7fa2-163a-40bd-b78a-ff37f5eade21 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:13:59.960763 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e7fa2-163a-40bd-b78a-ff37f5eade21 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:13:59.961086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e7fa2-163a-40bd-b78a-ff37f5eade21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:13:59.989128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-082e7fa2-163a-40bd-b78a-ff37f5eade21 None None] MFA Rules not processed for user `7519ef837c7848158ee0b12e663ddab9`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:13:59.990443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-082e7fa2-163a-40bd-b78a-ff37f5eade21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:00.053930 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1445/2840] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:13:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:00.066165 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235a7ed-ee41-44aa-9cff-b02bf4ec5814 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:00.067923 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-7235a7ed-ee41-44aa-9cff-b02bf4ec5814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:00.128437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235a7ed-ee41-44aa-9cff-b02bf4ec5814 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:00.132293 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c852f-3d9a-4664-95e5-1142e749ca15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:00.133008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c852f-3d9a-4664-95e5-1142e749ca15 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:00.133294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c852f-3d9a-4664-95e5-1142e749ca15 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:00.133573 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c852f-3d9a-4664-95e5-1142e749ca15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:00.142701 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c852f-3d9a-4664-95e5-1142e749ca15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:00.144159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c852f-3d9a-4664-95e5-1142e749ca15 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:00.153627 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1396/2841] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:14:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:14:00.174342 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e7fa2-163a-40bd-b78a-ff37f5eade21 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:00.241782 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e7fa2-163a-40bd-b78a-ff37f5eade21 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:00.246660 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:00.248507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:00.248507 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:00.248896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] PATH_INFO: `/v3/users/7519ef837c7848158ee0b12e663ddab9` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:00.260328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7519ef837c7848158ee0b12e663ddab9)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:00.260328 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:00.517729 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7519ef837c7848158ee0b12e663ddab9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:00.531285 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7519ef837c7848158ee0b12e663ddab9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:00.538398 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7519ef837c7848158ee0b12e663ddab9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:00.539195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7519ef837c7848158ee0b12e663ddab9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:00.632919 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7519ef837c7848158ee0b12e663ddab9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:00.635417 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:00.640087 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1446/2842] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:14:00 2022] DELETE /identity/v3/users/7519ef837c7848158ee0b12e663ddab9 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:14:00.645034 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f030a9-becc-4b81-b0c7-4dcb0b87a649 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:00.645325 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f030a9-becc-4b81-b0c7-4dcb0b87a649 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:00.645606 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f030a9-becc-4b81-b0c7-4dcb0b87a649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:00.663851 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-85f030a9-becc-4b81-b0c7-4dcb0b87a649 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:00.667355 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-85f030a9-becc-4b81-b0c7-4dcb0b87a649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:00.728635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1397/2843] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:00.741899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:00.741899 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:00.799625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc673a-4614-436a-993c-b8cc2f1c4a9c admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:00.803195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8685dba-d918-4fc5-a43a-f7d461678ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:00.803850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8685dba-d918-4fc5-a43a-f7d461678ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:00.804141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8685dba-d918-4fc5-a43a-f7d461678ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:00.804424 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8685dba-d918-4fc5-a43a-f7d461678ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:00.813408 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8685dba-d918-4fc5-a43a-f7d461678ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:00.814844 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8685dba-d918-4fc5-a43a-f7d461678ff2 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:00.824678 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1447/2844] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:14:01.263098 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f030a9-becc-4b81-b0c7-4dcb0b87a649 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:01.264559 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-85f030a9-becc-4b81-b0c7-4dcb0b87a649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:01.277978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f030a9-becc-4b81-b0c7-4dcb0b87a649 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:01.283557 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:01.284454 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:01.284769 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:01.285050 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] PATH_INFO: `/v3/projects/1969f5fd958e4f9ab887aa944f91028c` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:01.286273 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1969f5fd958e4f9ab887aa944f91028c)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:01.288197 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:01.391699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:01.400596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1969f5fd958e4f9ab887aa944f91028c', 'request_id': 'req-e25eb3b8-4161-4f83-942e-7da3ff5faad8'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:01.407253 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1969f5fd958e4f9ab887aa944f91028c', 'request_id': 'req-e25eb3b8-4161-4f83-942e-7da3ff5faad8'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:01.484411 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1969f5fd958e4f9ab887aa944f91028c', 'request_id': 'req-e25eb3b8-4161-4f83-942e-7da3ff5faad8'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:01.495755 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] The token cache is being invalidate because project 1969f5fd958e4f9ab887aa944f91028c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:14:01.496055 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:01.497232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-e25eb3b8-4161-4f83-942e-7da3ff5faad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:01.499431 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1398/2845] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:14:01 2022] DELETE /identity/v3/projects/1969f5fd958e4f9ab887aa944f91028c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:14:01.504358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13b0bc-32d1-41ab-8cf0-b1acdd99400b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:01.504695 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13b0bc-32d1-41ab-8cf0-b1acdd99400b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:01.504980 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13b0bc-32d1-41ab-8cf0-b1acdd99400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:01.523124 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-2f13b0bc-32d1-41ab-8cf0-b1acdd99400b None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:01.524291 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-2f13b0bc-32d1-41ab-8cf0-b1acdd99400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:01.583248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1448/2846] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:14:01.592008 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099f1e5-8c25-4a88-b12f-83bbb857b83f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:01.592305 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099f1e5-8c25-4a88-b12f-83bbb857b83f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:01.592589 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099f1e5-8c25-4a88-b12f-83bbb857b83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:01.611477 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-3099f1e5-8c25-4a88-b12f-83bbb857b83f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:01.612709 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-3099f1e5-8c25-4a88-b12f-83bbb857b83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:01.620631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1399/2847] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:01.625400 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13b0bc-32d1-41ab-8cf0-b1acdd99400b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:01.635121 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13b0bc-32d1-41ab-8cf0-b1acdd99400b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:01.640316 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc19d4-766b-4d1b-918e-b796474ee6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:01.641078 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc19d4-766b-4d1b-918e-b796474ee6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:01.641369 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc19d4-766b-4d1b-918e-b796474ee6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:01.641686 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc19d4-766b-4d1b-918e-b796474ee6b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:01.642461 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc19d4-766b-4d1b-918e-b796474ee6b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:14:01.643289 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc19d4-766b-4d1b-918e-b796474ee6b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:01.644484 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc19d4-766b-4d1b-918e-b796474ee6b6 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:01.656995 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1449/2848] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:14:01.664282 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099f1e5-8c25-4a88-b12f-83bbb857b83f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:01.674238 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099f1e5-8c25-4a88-b12f-83bbb857b83f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:01.678978 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4caf92-816f-408e-a178-7aa134c68032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:01.684151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4caf92-816f-408e-a178-7aa134c68032 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:01.684151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4caf92-816f-408e-a178-7aa134c68032 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:01.684151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4caf92-816f-408e-a178-7aa134c68032 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:01.684151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4caf92-816f-408e-a178-7aa134c68032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:01.684151 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4caf92-816f-408e-a178-7aa134c68032 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:01.749351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1400/2849] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:14:01 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:14:01.753691 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc19d4-766b-4d1b-918e-b796474ee6b6 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:01.764049 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc19d4-766b-4d1b-918e-b796474ee6b6 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:01.825602 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692df6d-4622-46c5-bcfc-545150cde60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:01.826254 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692df6d-4622-46c5-bcfc-545150cde60b admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:01.826543 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692df6d-4622-46c5-bcfc-545150cde60b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:01.826825 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692df6d-4622-46c5-bcfc-545150cde60b admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:01.827530 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692df6d-4622-46c5-bcfc-545150cde60b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:01.829492 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692df6d-4622-46c5-bcfc-545150cde60b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:01.925427 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1450/2850] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:14:01 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:14:01.929631 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4caf92-816f-408e-a178-7aa134c68032 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:01.939873 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4caf92-816f-408e-a178-7aa134c68032 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:01.944822 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a21225-9334-4b94-bb8a-f55dbba02586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:01.945485 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a21225-9334-4b94-bb8a-f55dbba02586 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:01.945765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a21225-9334-4b94-bb8a-f55dbba02586 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:01.946039 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a21225-9334-4b94-bb8a-f55dbba02586 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:01.946763 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a21225-9334-4b94-bb8a-f55dbba02586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:01.948343 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a21225-9334-4b94-bb8a-f55dbba02586 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:01.959180 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1401/2851] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 24 20:14:01.963370 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692df6d-4622-46c5-bcfc-545150cde60b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:01.974011 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692df6d-4622-46c5-bcfc-545150cde60b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:01.978791 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c2d69-55cb-47ff-8dff-c4443b5546f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:01.979457 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c2d69-55cb-47ff-8dff-c4443b5546f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:01.979750 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c2d69-55cb-47ff-8dff-c4443b5546f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:01.980037 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c2d69-55cb-47ff-8dff-c4443b5546f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:01.980814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c2d69-55cb-47ff-8dff-c4443b5546f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:01.983088 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c2d69-55cb-47ff-8dff-c4443b5546f1 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:01.993361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1451/2852] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:14:01.997647 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a21225-9334-4b94-bb8a-f55dbba02586 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:02.007696 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a21225-9334-4b94-bb8a-f55dbba02586 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:02.012323 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d6b01-9b24-4ada-b712-aa85965afd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:02.013235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6b01-9b24-4ada-b712-aa85965afd1f admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:02.013527 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6b01-9b24-4ada-b712-aa85965afd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:02.013854 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6b01-9b24-4ada-b712-aa85965afd1f admin admin] PATH_INFO: `/v3/projects/2ee96219ba494bb2af0e3d6111fa73c5/users/c5095c7fbbfe4dc5a91a38db766051e9/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:02.029145 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d6b01-9b24-4ada-b712-aa85965afd1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee96219ba494bb2af0e3d6111fa73c5, user_id=c5095c7fbbfe4dc5a91a38db766051e9, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:02.031847 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d6b01-9b24-4ada-b712-aa85965afd1f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:02.124733 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1402/2853] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:14:01 2022] PUT /identity/v3/projects/2ee96219ba494bb2af0e3d6111fa73c5/users/c5095c7fbbfe4dc5a91a38db766051e9/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:14:02.186953 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eec11-9f9d-4014-9ef4-be8009b5870b None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:02.187248 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eec11-9f9d-4014-9ef4-be8009b5870b None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:02.187533 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eec11-9f9d-4014-9ef4-be8009b5870b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:02.206257 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:02.214772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f71eec11-9f9d-4014-9ef4-be8009b5870b None None] MFA Rules not processed for user `c5095c7fbbfe4dc5a91a38db766051e9`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:02.217752 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f71eec11-9f9d-4014-9ef4-be8009b5870b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:02.282349 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1452/2854] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:02.293418 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d6b01-9b24-4ada-b712-aa85965afd1f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:02.295317 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-c75d6b01-9b24-4ada-b712-aa85965afd1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:02.351645 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d6b01-9b24-4ada-b712-aa85965afd1f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:02.355235 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e34cea-4620-48b9-b32f-f93191661d36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:02.355874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e34cea-4620-48b9-b32f-f93191661d36 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:02.356159 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e34cea-4620-48b9-b32f-f93191661d36 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:02.356437 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e34cea-4620-48b9-b32f-f93191661d36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:02.364772 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e34cea-4620-48b9-b32f-f93191661d36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:02.366190 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e34cea-4620-48b9-b32f-f93191661d36 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:02.374903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1403/2855] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:14:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:14:02.398888 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71eec11-9f9d-4014-9ef4-be8009b5870b None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:02.462575 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71eec11-9f9d-4014-9ef4-be8009b5870b None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:02.467843 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:02.468745 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:02.469034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:02.469313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] PATH_INFO: `/v3/users/c5095c7fbbfe4dc5a91a38db766051e9` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:02.470515 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5095c7fbbfe4dc5a91a38db766051e9)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:02.472449 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:02.679806 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5095c7fbbfe4dc5a91a38db766051e9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:02.692101 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5095c7fbbfe4dc5a91a38db766051e9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:02.699185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5095c7fbbfe4dc5a91a38db766051e9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:02.700062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5095c7fbbfe4dc5a91a38db766051e9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:02.753031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5095c7fbbfe4dc5a91a38db766051e9'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:02.759666 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1453/2856] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:14:02 2022] DELETE /identity/v3/users/c5095c7fbbfe4dc5a91a38db766051e9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:14:02.765009 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7a2a7-a2ef-4605-bd42-fa98f9a2a2ea None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:02.765303 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7a2a7-a2ef-4605-bd42-fa98f9a2a2ea None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:02.765585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7a2a7-a2ef-4605-bd42-fa98f9a2a2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:02.783139 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-69e7a2a7-a2ef-4605-bd42-fa98f9a2a2ea None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:02.784387 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-69e7a2a7-a2ef-4605-bd42-fa98f9a2a2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:02.845845 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1404/2857] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:02.857317 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:02.858773 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:02.880466 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:02.924700 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b26ba4-4fa2-4519-827e-9d72310e9f1b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:02.928579 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8aeb9-4000-4095-8f64-9602ec61f9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:02.929278 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8aeb9-4000-4095-8f64-9602ec61f9db service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:02.929573 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8aeb9-4000-4095-8f64-9602ec61f9db service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:02.929975 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8aeb9-4000-4095-8f64-9602ec61f9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:02.939906 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8aeb9-4000-4095-8f64-9602ec61f9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:02.942137 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8aeb9-4000-4095-8f64-9602ec61f9db service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:02.953141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1454/2858] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:14:03.348529 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7a2a7-a2ef-4605-bd42-fa98f9a2a2ea None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:03.350080 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-69e7a2a7-a2ef-4605-bd42-fa98f9a2a2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:03.359031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:03.363977 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7a2a7-a2ef-4605-bd42-fa98f9a2a2ea None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:03.369786 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:03.370578 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:03.370866 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:03.371138 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] PATH_INFO: `/v3/projects/2ee96219ba494bb2af0e3d6111fa73c5` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:03.372382 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee96219ba494bb2af0e3d6111fa73c5)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:03.375127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:03.522831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee96219ba494bb2af0e3d6111fa73c5', 'request_id': 'req-bee6cd26-0e3b-4372-afcb-4276acb8f774'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:03.529496 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee96219ba494bb2af0e3d6111fa73c5', 'request_id': 'req-bee6cd26-0e3b-4372-afcb-4276acb8f774'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:03.598853 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee96219ba494bb2af0e3d6111fa73c5', 'request_id': 'req-bee6cd26-0e3b-4372-afcb-4276acb8f774'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:03.609823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] The token cache is being invalidate because project 2ee96219ba494bb2af0e3d6111fa73c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:14:03.610116 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:03.610742 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-bee6cd26-0e3b-4372-afcb-4276acb8f774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:03.612285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1405/2859] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:14:03 2022] DELETE /identity/v3/projects/2ee96219ba494bb2af0e3d6111fa73c5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:14:03.617406 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2b2c5-c90a-43b3-883e-ccebc7dde1a2 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:03.617707 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2b2c5-c90a-43b3-883e-ccebc7dde1a2 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:03.619061 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2b2c5-c90a-43b3-883e-ccebc7dde1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:03.636007 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-65b2b2c5-c90a-43b3-883e-ccebc7dde1a2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:03.637294 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-65b2b2c5-c90a-43b3-883e-ccebc7dde1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:03.696086 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1455/2860] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:03.703714 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b65b9-824c-4f8a-9474-16246bc5df7f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:03.704006 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b65b9-824c-4f8a-9474-16246bc5df7f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:03.704285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b65b9-824c-4f8a-9474-16246bc5df7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:03.723086 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-319b65b9-824c-4f8a-9474-16246bc5df7f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:03.724294 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-319b65b9-824c-4f8a-9474-16246bc5df7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:03.732955 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1406/2861] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:03.737274 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2b2c5-c90a-43b3-883e-ccebc7dde1a2 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:03.746927 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2b2c5-c90a-43b3-883e-ccebc7dde1a2 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:03.751490 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98662308-5dcc-4fd6-b0a0-4f26924cac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:03.752145 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98662308-5dcc-4fd6-b0a0-4f26924cac4d admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:03.752438 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98662308-5dcc-4fd6-b0a0-4f26924cac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:03.752749 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98662308-5dcc-4fd6-b0a0-4f26924cac4d admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:03.753477 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98662308-5dcc-4fd6-b0a0-4f26924cac4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:14:03.753787 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98662308-5dcc-4fd6-b0a0-4f26924cac4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:03.755377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98662308-5dcc-4fd6-b0a0-4f26924cac4d admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:03.767373 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1456/2862] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:14:03.771556 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b65b9-824c-4f8a-9474-16246bc5df7f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:03.781734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b65b9-824c-4f8a-9474-16246bc5df7f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:03.786391 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a74df-9969-492a-9954-c5e693002efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:03.787044 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a74df-9969-492a-9954-c5e693002efb admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:03.787338 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a74df-9969-492a-9954-c5e693002efb admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:03.787618 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a74df-9969-492a-9954-c5e693002efb admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:03.788316 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a74df-9969-492a-9954-c5e693002efb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:03.790263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a74df-9969-492a-9954-c5e693002efb admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:03.914765 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1407/2863] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:14:03 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:14:03.919062 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98662308-5dcc-4fd6-b0a0-4f26924cac4d admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:03.929315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98662308-5dcc-4fd6-b0a0-4f26924cac4d admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:03.984902 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297872d-d77d-4843-904b-e3f1325d43c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:03.985572 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297872d-d77d-4843-904b-e3f1325d43c0 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:03.985894 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297872d-d77d-4843-904b-e3f1325d43c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:03.986134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297872d-d77d-4843-904b-e3f1325d43c0 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:03.986816 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297872d-d77d-4843-904b-e3f1325d43c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:03.988767 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297872d-d77d-4843-904b-e3f1325d43c0 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:04.079511 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1457/2864] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:14:03 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:14:04.083886 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a74df-9969-492a-9954-c5e693002efb admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:04.095015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a74df-9969-492a-9954-c5e693002efb admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:04.099635 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a746c07-2425-4415-80b1-4b8a25f415f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:04.100315 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a746c07-2425-4415-80b1-4b8a25f415f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:04.100596 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a746c07-2425-4415-80b1-4b8a25f415f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:04.100915 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a746c07-2425-4415-80b1-4b8a25f415f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:04.101642 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a746c07-2425-4415-80b1-4b8a25f415f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:04.103232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a746c07-2425-4415-80b1-4b8a25f415f3 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:04.114911 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1408/2865] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:14:04.119114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297872d-d77d-4843-904b-e3f1325d43c0 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:04.129972 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297872d-d77d-4843-904b-e3f1325d43c0 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:04.134952 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c80a9a-36b6-414d-b06d-a14ed0b1e3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:04.135638 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c80a9a-36b6-414d-b06d-a14ed0b1e3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:04.135926 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c80a9a-36b6-414d-b06d-a14ed0b1e3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:04.136204 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c80a9a-36b6-414d-b06d-a14ed0b1e3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:04.136983 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c80a9a-36b6-414d-b06d-a14ed0b1e3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:04.139167 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c80a9a-36b6-414d-b06d-a14ed0b1e3d3 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:04.150139 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1458/2866] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:14:04.154715 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a746c07-2425-4415-80b1-4b8a25f415f3 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:04.165040 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a746c07-2425-4415-80b1-4b8a25f415f3 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:04.170285 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca16185-5f76-4e6e-a09b-39a90d2a11b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:04.171182 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca16185-5f76-4e6e-a09b-39a90d2a11b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:04.171466 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca16185-5f76-4e6e-a09b-39a90d2a11b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:04.171748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca16185-5f76-4e6e-a09b-39a90d2a11b6 admin admin] PATH_INFO: `/v3/projects/4c174447863a4673b5d751fc2319ea77/users/c4c10ed0e50f4f1288bdfc7fa07c9339/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:04.188015 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca16185-5f76-4e6e-a09b-39a90d2a11b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c174447863a4673b5d751fc2319ea77, user_id=c4c10ed0e50f4f1288bdfc7fa07c9339, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:04.190954 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca16185-5f76-4e6e-a09b-39a90d2a11b6 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:04.302211 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1409/2867] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:14:04 2022] PUT /identity/v3/projects/4c174447863a4673b5d751fc2319ea77/users/c4c10ed0e50f4f1288bdfc7fa07c9339/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 24 20:14:04.314324 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bb48e-6818-4c9a-9383-998919bad77d None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:04.316224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bb48e-6818-4c9a-9383-998919bad77d None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:04.316224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bb48e-6818-4c9a-9383-998919bad77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:04.342641 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-973bb48e-6818-4c9a-9383-998919bad77d None None] MFA Rules not processed for user `c4c10ed0e50f4f1288bdfc7fa07c9339`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:04.343804 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-973bb48e-6818-4c9a-9383-998919bad77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:04.356307 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:04.407595 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1459/2868] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:04.418202 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca16185-5f76-4e6e-a09b-39a90d2a11b6 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:04.419874 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-0ca16185-5f76-4e6e-a09b-39a90d2a11b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:04.478565 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca16185-5f76-4e6e-a09b-39a90d2a11b6 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:04.482748 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723916b-3719-4cfe-b3c8-1b6f26092821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:04.483392 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723916b-3719-4cfe-b3c8-1b6f26092821 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:04.483675 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723916b-3719-4cfe-b3c8-1b6f26092821 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:04.483947 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723916b-3719-4cfe-b3c8-1b6f26092821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:04.492393 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723916b-3719-4cfe-b3c8-1b6f26092821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:04.493844 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723916b-3719-4cfe-b3c8-1b6f26092821 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:04.503174 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1410/2869] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:14:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:14:14.472439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=20 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:14.499734 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:15.637694 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bb48e-6818-4c9a-9383-998919bad77d None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:15.702499 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bb48e-6818-4c9a-9383-998919bad77d None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:15.707358 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:15.708223 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:15.708508 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:15.708819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] PATH_INFO: `/v3/users/c4c10ed0e50f4f1288bdfc7fa07c9339` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:15.710004 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c10ed0e50f4f1288bdfc7fa07c9339)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:15.711920 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:15.943343 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c10ed0e50f4f1288bdfc7fa07c9339'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:15.955301 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c10ed0e50f4f1288bdfc7fa07c9339'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:15.963031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c10ed0e50f4f1288bdfc7fa07c9339'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:15.967016 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c10ed0e50f4f1288bdfc7fa07c9339'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:16.048374 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c10ed0e50f4f1288bdfc7fa07c9339'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:16.054535 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1460/2870] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:14:15 2022] DELETE /identity/v3/users/c4c10ed0e50f4f1288bdfc7fa07c9339 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:14:16.059221 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418de6d-438f-48cb-9ceb-07a10d5789e2 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:16.060031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418de6d-438f-48cb-9ceb-07a10d5789e2 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:16.060031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418de6d-438f-48cb-9ceb-07a10d5789e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:16.077525 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-2418de6d-438f-48cb-9ceb-07a10d5789e2 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:16.078818 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2418de6d-438f-48cb-9ceb-07a10d5789e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:16.138880 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1411/2871] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:16.149097 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:16.150549 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:16.209391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a1bf4-54ac-4a75-84a3-e8b2d784cbcc admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:16.212883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e95bad-e2af-440e-92cb-d040c079ea89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:16.213537 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e95bad-e2af-440e-92cb-d040c079ea89 service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:16.213831 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e95bad-e2af-440e-92cb-d040c079ea89 service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:16.214114 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e95bad-e2af-440e-92cb-d040c079ea89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:16.223462 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e95bad-e2af-440e-92cb-d040c079ea89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:16.225003 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e95bad-e2af-440e-92cb-d040c079ea89 service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:16.234778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1461/2872] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:14:16.634248 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418de6d-438f-48cb-9ceb-07a10d5789e2 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:16.635936 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2418de6d-438f-48cb-9ceb-07a10d5789e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:16.649703 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418de6d-438f-48cb-9ceb-07a10d5789e2 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:16.654941 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:16.656434 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:16.657417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:16.657699 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] PATH_INFO: `/v3/projects/4c174447863a4673b5d751fc2319ea77` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:16.659087 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c174447863a4673b5d751fc2319ea77)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:16.661143 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:16.760184 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c174447863a4673b5d751fc2319ea77', 'request_id': 'req-193271ae-10b1-4977-b16c-773a000aabd6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:16.766827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c174447863a4673b5d751fc2319ea77', 'request_id': 'req-193271ae-10b1-4977-b16c-773a000aabd6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:16.809858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c174447863a4673b5d751fc2319ea77', 'request_id': 'req-193271ae-10b1-4977-b16c-773a000aabd6'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:16.821534 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] The token cache is being invalidate because project 4c174447863a4673b5d751fc2319ea77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:14:16.821816 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:16.822546 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-193271ae-10b1-4977-b16c-773a000aabd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:16.824127 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1412/2873] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:14:16 2022] DELETE /identity/v3/projects/4c174447863a4673b5d751fc2319ea77 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:14:16.829160 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d307b4-e0f0-438f-be7f-182e18f908e8 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:16.829696 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d307b4-e0f0-438f-be7f-182e18f908e8 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:16.829835 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d307b4-e0f0-438f-be7f-182e18f908e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:16.847603 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-e3d307b4-e0f0-438f-be7f-182e18f908e8 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:16.848896 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-e3d307b4-e0f0-438f-be7f-182e18f908e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:16.866817 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:16.913785 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1462/2874] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:16.922308 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93a43-7c48-4e7f-a342-f71f933ab292 None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:16.922638 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93a43-7c48-4e7f-a342-f71f933ab292 None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:16.922922 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93a43-7c48-4e7f-a342-f71f933ab292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:16.942178 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-eaa93a43-7c48-4e7f-a342-f71f933ab292 None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:16.943385 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-eaa93a43-7c48-4e7f-a342-f71f933ab292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:16.951378 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1413/2875] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:16.956192 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d307b4-e0f0-438f-be7f-182e18f908e8 None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:16.966128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d307b4-e0f0-438f-be7f-182e18f908e8 None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:16.971315 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedcf83-f13d-4797-b344-f882438edd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:16.971978 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedcf83-f13d-4797-b344-f882438edd26 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:16.972268 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedcf83-f13d-4797-b344-f882438edd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:16.972551 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedcf83-f13d-4797-b344-f882438edd26 admin admin] PATH_INFO: `/v3/domains` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:16.973325 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedcf83-f13d-4797-b344-f882438edd26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86807) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 24 20:14:16.973640 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedcf83-f13d-4797-b344-f882438edd26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:16.975240 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedcf83-f13d-4797-b344-f882438edd26 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:16.987534 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1463/2876] 10.222.0.17 () {58 vars in 1152 bytes} [Sun Jul 24 20:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 24 20:14:16.991820 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93a43-7c48-4e7f-a342-f71f933ab292 None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:17.001719 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93a43-7c48-4e7f-a342-f71f933ab292 None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:17.006417 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f17c-24cc-4624-b6d7-69c538714e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:17.007070 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f17c-24cc-4624-b6d7-69c538714e68 admin admin] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:17.007364 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f17c-24cc-4624-b6d7-69c538714e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:17.007644 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f17c-24cc-4624-b6d7-69c538714e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:17.008330 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43f17c-24cc-4624-b6d7-69c538714e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:17.010313 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43f17c-24cc-4624-b6d7-69c538714e68 admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:17.091351 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1414/2877] 10.222.0.17 () {60 vars in 1152 bytes} [Sun Jul 24 20:14:16 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 24 20:14:17.095531 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedcf83-f13d-4797-b344-f882438edd26 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:17.105869 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedcf83-f13d-4797-b344-f882438edd26 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:17.162361 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448543aa-a2bd-4dae-b959-145b8ff2dbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:17.163020 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448543aa-a2bd-4dae-b959-145b8ff2dbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:17.163303 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448543aa-a2bd-4dae-b959-145b8ff2dbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:17.163578 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448543aa-a2bd-4dae-b959-145b8ff2dbc7 admin admin] PATH_INFO: `/v3/users` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:17.164498 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448543aa-a2bd-4dae-b959-145b8ff2dbc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:17.166441 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448543aa-a2bd-4dae-b959-145b8ff2dbc7 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:17.250180 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1464/2878] 10.222.0.17 () {60 vars in 1143 bytes} [Sun Jul 24 20:14:17 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 24 20:14:17.254531 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f17c-24cc-4624-b6d7-69c538714e68 admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:17.265310 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f17c-24cc-4624-b6d7-69c538714e68 admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:17.269966 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d98e8-787c-4e10-81fd-0581072aaa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:17.274920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d98e8-787c-4e10-81fd-0581072aaa3a admin admin] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:17.274920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d98e8-787c-4e10-81fd-0581072aaa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:17.274920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d98e8-787c-4e10-81fd-0581072aaa3a admin admin] PATH_INFO: `/v3/roles` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:17.274920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d98e8-787c-4e10-81fd-0581072aaa3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:17.274920 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d98e8-787c-4e10-81fd-0581072aaa3a admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:17.284057 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1415/2879] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:14:17.288368 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448543aa-a2bd-4dae-b959-145b8ff2dbc7 admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:17.299031 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448543aa-a2bd-4dae-b959-145b8ff2dbc7 admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:17.303715 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d38c5-584b-40a5-b002-86d830a1c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:17.304377 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d38c5-584b-40a5-b002-86d830a1c452 admin admin] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:17.304760 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d38c5-584b-40a5-b002-86d830a1c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:17.305053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d38c5-584b-40a5-b002-86d830a1c452 admin admin] PATH_INFO: `/v3/roles` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:17.305784 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d38c5-584b-40a5-b002-86d830a1c452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:17.307389 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d38c5-584b-40a5-b002-86d830a1c452 admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:17.318778 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1465/2880] 10.222.0.17 () {58 vars in 1121 bytes} [Sun Jul 24 20:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 24 20:14:17.323032 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d98e8-787c-4e10-81fd-0581072aaa3a admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:17.333678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d98e8-787c-4e10-81fd-0581072aaa3a admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:17.338542 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466ba3c-5639-4356-a98f-64a5b9f7700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:17.339322 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466ba3c-5639-4356-a98f-64a5b9f7700f admin admin] REQUEST_METHOD: `PUT` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:17.339612 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466ba3c-5639-4356-a98f-64a5b9f7700f admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:17.339895 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466ba3c-5639-4356-a98f-64a5b9f7700f admin admin] PATH_INFO: `/v3/projects/ddf4bda527a74e9e86385f49233694a2/users/21f941952c3e41eebed90700b5bedbdb/roles/2a20315693ce44d696f86173c0f6eb09` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:17.355047 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466ba3c-5639-4356-a98f-64a5b9f7700f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf4bda527a74e9e86385f49233694a2, user_id=21f941952c3e41eebed90700b5bedbdb, role_id=2a20315693ce44d696f86173c0f6eb09)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:17.357730 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466ba3c-5639-4356-a98f-64a5b9f7700f admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:17.433271 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1416/2881] 10.222.0.17 () {60 vars in 1482 bytes} [Sun Jul 24 20:14:17 2022] PUT /identity/v3/projects/ddf4bda527a74e9e86385f49233694a2/users/21f941952c3e41eebed90700b5bedbdb/roles/2a20315693ce44d696f86173c0f6eb09 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:14:17.444999 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb32ca8-6345-409f-8671-43948ac0ffef None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:17.445306 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb32ca8-6345-409f-8671-43948ac0ffef None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:17.446974 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb32ca8-6345-409f-8671-43948ac0ffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:17.474795 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-ebb32ca8-6345-409f-8671-43948ac0ffef None None] MFA Rules not processed for user `21f941952c3e41eebed90700b5bedbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:17.476015 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ebb32ca8-6345-409f-8671-43948ac0ffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:17.540209 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1466/2882] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:17.551268 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466ba3c-5639-4356-a98f-64a5b9f7700f admin admin] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:17.552994 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-2466ba3c-5639-4356-a98f-64a5b9f7700f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:17.575603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:17.611445 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466ba3c-5639-4356-a98f-64a5b9f7700f admin admin] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:17.615410 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6fe8-880f-4257-b8fe-bb967b797ca8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe7a67bd815147afa3f186e95b004469', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:17.616407 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6fe8-880f-4257-b8fe-bb967b797ca8 service cinder] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:17.616783 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6fe8-880f-4257-b8fe-bb967b797ca8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:17.617062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6fe8-880f-4257-b8fe-bb967b797ca8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:17.625676 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6fe8-880f-4257-b8fe-bb967b797ca8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:17.627153 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6fe8-880f-4257-b8fe-bb967b797ca8 service cinder] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:17.636036 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1417/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:14:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:14:19.349134 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f713649-b1cc-48ec-97b8-2f35135313ac None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:19.349665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f713649-b1cc-48ec-97b8-2f35135313ac None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:19.349665 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f713649-b1cc-48ec-97b8-2f35135313ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:19.366814 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-8f713649-b1cc-48ec-97b8-2f35135313ac None None] MFA Rules not processed for user `21f941952c3e41eebed90700b5bedbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:19.367996 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-8f713649-b1cc-48ec-97b8-2f35135313ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:19.375332 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1467/2884] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:21.272929 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6fe8-880f-4257-b8fe-bb967b797ca8 service cinder] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:21.274400 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-815c6fe8-880f-4257-b8fe-bb967b797ca8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:21.353574 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6fe8-880f-4257-b8fe-bb967b797ca8 service cinder] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:21.360588 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:21.361333 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:21.361623 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:21.361903 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:21.364088 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:21.377610 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:21.379212 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:21.395563 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1418/2885] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:14:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:14:29.362128 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:31.386290 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=49 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:35.558426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9f62b-72af-49e4-9cec-2a798e3b8892 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:35.559033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9f62b-72af-49e4-9cec-2a798e3b8892 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:35.559033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9f62b-72af-49e4-9cec-2a798e3b8892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:35.607802 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-bcc9f62b-72af-49e4-9cec-2a798e3b8892 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:35.609127 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-bcc9f62b-72af-49e4-9cec-2a798e3b8892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:35.675525 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1468/2886] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:14:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 24 20:14:35.840386 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:35.842042 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:35.941787 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae439134-b47d-4c3c-b009-a907ef9f63d7 service placement] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:35.947196 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a7eb5-d230-4504-ae8e-f3ad8f33f3c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:35.949747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7eb5-d230-4504-ae8e-f3ad8f33f3c2 service swift] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:35.949747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7eb5-d230-4504-ae8e-f3ad8f33f3c2 service swift] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:35.949747 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7eb5-d230-4504-ae8e-f3ad8f33f3c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:35.958318 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a7eb5-d230-4504-ae8e-f3ad8f33f3c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:35.959925 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a7eb5-d230-4504-ae8e-f3ad8f33f3c2 service swift] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:35.969123 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1419/2887] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:14:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:14:36.802350 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e3ddd-15d7-4729-a231-3cd8a970f230 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:36.802912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e3ddd-15d7-4729-a231-3cd8a970f230 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:36.802912 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e3ddd-15d7-4729-a231-3cd8a970f230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:36.823574 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-f98e3ddd-15d7-4729-a231-3cd8a970f230 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:36.824850 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-f98e3ddd-15d7-4729-a231-3cd8a970f230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:36.832185 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1469/2888] 10.222.0.17 () {58 vars in 967 bytes} [Sun Jul 24 20:14:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:36.844432 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a7eb5-d230-4504-ae8e-f3ad8f33f3c2 service swift] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:36.855360 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a7eb5-d230-4504-ae8e-f3ad8f33f3c2 service swift] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:36.861858 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a930d55-8f8e-42da-93f6-65a6b5d0197b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:36.863381 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a930d55-8f8e-42da-93f6-65a6b5d0197b service swift] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:36.863798 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a930d55-8f8e-42da-93f6-65a6b5d0197b service swift] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:36.864213 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a930d55-8f8e-42da-93f6-65a6b5d0197b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:36.873465 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a930d55-8f8e-42da-93f6-65a6b5d0197b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:36.875027 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a930d55-8f8e-42da-93f6-65a6b5d0197b service swift] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:36.885640 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1420/2889] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:14:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:14:37.605267 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3a17d-7e7c-4868-a2f9-dbe901496028 None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:37.605721 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3a17d-7e7c-4868-a2f9-dbe901496028 None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:37.606104 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3a17d-7e7c-4868-a2f9-dbe901496028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:37.625055 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-0cd3a17d-7e7c-4868-a2f9-dbe901496028 None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:37.626305 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0cd3a17d-7e7c-4868-a2f9-dbe901496028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:37.633819 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1470/2890] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Jul 24 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:37.646301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a930d55-8f8e-42da-93f6-65a6b5d0197b service swift] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:37.657154 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a930d55-8f8e-42da-93f6-65a6b5d0197b service swift] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:37.662823 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173ea6d-fd72-42a1-bcf1-b1545bac87c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:37.663473 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173ea6d-fd72-42a1-bcf1-b1545bac87c6 service swift] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:37.663760 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173ea6d-fd72-42a1-bcf1-b1545bac87c6 service swift] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:37.664038 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173ea6d-fd72-42a1-bcf1-b1545bac87c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:37.672585 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173ea6d-fd72-42a1-bcf1-b1545bac87c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:37.674178 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173ea6d-fd72-42a1-bcf1-b1545bac87c6 service swift] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:37.682603 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1421/2891] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:14:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:14:38.270053 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3720-4cb9-44e2-a1b5-3deb27e7d4eb None None] REQUEST_METHOD: `POST` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:38.270522 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3720-4cb9-44e2-a1b5-3deb27e7d4eb None None] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:38.270634 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3720-4cb9-44e2-a1b5-3deb27e7d4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:38.288673 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.auth.core [None req-7f4f3720-4cb9-44e2-a1b5-3deb27e7d4eb None None] MFA Rules not processed for user `8fb7202aa94640129f33a310f6d00876`. Rule list: `[]` (Enabled: `True`). {{(pid=86807) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:38.290224 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-7f4f3720-4cb9-44e2-a1b5-3deb27e7d4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:38.297178 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1471/2892] 10.222.0.17 () {58 vars in 967 bytes} [Sun Jul 24 20:14:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:38.309292 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173ea6d-fd72-42a1-bcf1-b1545bac87c6 service swift] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:38.321678 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173ea6d-fd72-42a1-bcf1-b1545bac87c6 service swift] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:38.327134 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb6b1f-077e-4dc1-8e74-83e475f12a9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0849f31902da4613babb10a76c4548f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:38.327806 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6b1f-077e-4dc1-8e74-83e475f12a9e service swift] REQUEST_METHOD: `GET` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:38.328099 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6b1f-077e-4dc1-8e74-83e475f12a9e service swift] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:38.328374 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6b1f-077e-4dc1-8e74-83e475f12a9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:38.337625 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb6b1f-077e-4dc1-8e74-83e475f12a9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:38.339214 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb6b1f-077e-4dc1-8e74-83e475f12a9e service swift] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:38.348093 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1422/2893] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Jul 24 20:14:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 24 20:14:41.213951 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f3720-4cb9-44e2-a1b5-3deb27e7d4eb None None] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:41.276963 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f3720-4cb9-44e2-a1b5-3deb27e7d4eb None None] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:41.281433 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:41.282313 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:41.282601 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:41.282883 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] PATH_INFO: `/v3/users/21f941952c3e41eebed90700b5bedbdb` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:41.284106 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] RBAC: Authorizing `identity:delete_user(user_id=21f941952c3e41eebed90700b5bedbdb)` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:41.286625 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:41.474586 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21f941952c3e41eebed90700b5bedbdb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:41.479733 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:41.486443 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21f941952c3e41eebed90700b5bedbdb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:41.493439 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21f941952c3e41eebed90700b5bedbdb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:41.494213 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21f941952c3e41eebed90700b5bedbdb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:41.570446 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.notifications [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21f941952c3e41eebed90700b5bedbdb'} {{(pid=86807) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:41.577593 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1472/2894] 10.222.0.17 () {58 vars in 1223 bytes} [Sun Jul 24 20:14:41 2022] DELETE /identity/v3/users/21f941952c3e41eebed90700b5bedbdb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:14:41.582232 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018d76f-169a-4e12-baeb-6388e3c3658f None None] REQUEST_METHOD: `POST` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:41.582827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018d76f-169a-4e12-baeb-6388e3c3658f None None] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:41.582827 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018d76f-169a-4e12-baeb-6388e3c3658f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:41.603778 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.auth.core [None req-6018d76f-169a-4e12-baeb-6388e3c3658f None None] MFA Rules not processed for user `3cfc6b9179e849cdbe9d2d91ab4d0972`. Rule list: `[]` (Enabled: `True`). {{(pid=86806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 24 20:14:41.605031 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6018d76f-169a-4e12-baeb-6388e3c3658f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:41.672082 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1423/2895] 10.222.0.17 () {58 vars in 957 bytes} [Sun Jul 24 20:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 24 20:14:41.681966 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:41.683435 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:41.742354 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53fcbd-52cb-458c-ac7c-079dd9df975b admin admin] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:41.746141 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f392f-2e7d-4f80-bd2c-2edc06e02f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9763f9899214df3929f3816f009e857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:41.746797 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f392f-2e7d-4f80-bd2c-2edc06e02f7b service neutron] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:41.747083 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f392f-2e7d-4f80-bd2c-2edc06e02f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:41.747363 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f392f-2e7d-4f80-bd2c-2edc06e02f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:41.757323 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f392f-2e7d-4f80-bd2c-2edc06e02f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:41.758772 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f392f-2e7d-4f80-bd2c-2edc06e02f7b service neutron] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:41.769833 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1473/2896] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:14:42.073692 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018d76f-169a-4e12-baeb-6388e3c3658f None None] Authenticating user token {{(pid=86806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:14:42.075141 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.fernet_utils [None req-6018d76f-169a-4e12-baeb-6388e3c3658f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:14:42.089943 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018d76f-169a-4e12-baeb-6388e3c3658f None None] Validating token access rules against request {{(pid=86806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:14:42.095172 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfc6b9179e849cdbe9d2d91ab4d0972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '245371efe76c48ce9eb35e790603face', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:14:42.096263 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:14:42.096550 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] SCRIPT_NAME: `/identity` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:14:42.096869 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] PATH_INFO: `/v3/projects/ddf4bda527a74e9e86385f49233694a2` {{(pid=86806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:14:42.098176 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddf4bda527a74e9e86385f49233694a2)` {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:14:42.100187 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] RBAC: Authorization granted {{(pid=86806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:14:42.211605 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:42.230301 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddf4bda527a74e9e86385f49233694a2', 'request_id': 'req-1723a851-e464-4047-a3b7-dfea20b2f02d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:42.237105 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddf4bda527a74e9e86385f49233694a2', 'request_id': 'req-1723a851-e464-4047-a3b7-dfea20b2f02d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:42.340831 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddf4bda527a74e9e86385f49233694a2', 'request_id': 'req-1723a851-e464-4047-a3b7-dfea20b2f02d'} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:42.352104 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] The token cache is being invalidate because project ddf4bda527a74e9e86385f49233694a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 24 20:14:42.352399 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:42.353062 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG keystone.notifications [None req-1723a851-e464-4047-a3b7-dfea20b2f02d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 24 20:14:42.354624 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: [pid: 86806|app: 0|req: 1424/2897] 10.222.0.17 () {58 vars in 1232 bytes} [Sun Jul 24 20:14:42 2022] DELETE /identity/v3/projects/ddf4bda527a74e9e86385f49233694a2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 24 20:14:51.765034 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=36,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:14:52.381766 dv-cinder-850449-2-iscsi devstack@keystone.service[86806]: DEBUG dbcounter [-] [86806] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=86806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 20:19:26.693150 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f392f-2e7d-4f80-bd2c-2edc06e02f7b service neutron] Authenticating user token {{(pid=86807) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 24 20:19:26.694548 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-1c1f392f-2e7d-4f80-bd2c-2edc06e02f7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:19:26.834096 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f392f-2e7d-4f80-bd2c-2edc06e02f7b service neutron] Validating token access rules against request {{(pid=86807) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 24 20:19:26.891545 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5aadf-ec56-415b-b887-51c25c2c5fdb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c1b23b83544423abc6919d805916b8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ee9dcb1ea40484080c0aeb065c702e9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86807) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 24 20:19:26.892195 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5aadf-ec56-415b-b887-51c25c2c5fdb service placement] REQUEST_METHOD: `GET` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 24 20:19:26.892510 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5aadf-ec56-415b-b887-51c25c2c5fdb service placement] SCRIPT_NAME: `/identity` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 24 20:19:26.892826 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5aadf-ec56-415b-b887-51c25c2c5fdb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86807) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 24 20:19:26.894583 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.fernet_utils [None req-0ad5aadf-ec56-415b-b887-51c25c2c5fdb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86807) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 24 20:19:26.905861 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5aadf-ec56-415b-b887-51c25c2c5fdb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 24 20:19:26.907426 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5aadf-ec56-415b-b887-51c25c2c5fdb service placement] RBAC: Authorization granted {{(pid=86807) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 24 20:19:26.917391 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: [pid: 86807|app: 0|req: 1474/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Jul 24 20:19:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 24 20:19:36.913033 dv-cinder-850449-2-iscsi devstack@keystone.service[86807]: DEBUG dbcounter [-] [86807] Writing DB stats keystone:SELECT=89 {{(pid=86807) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}